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Abstract. We deal with unweighted and weighted enumerations of lozenge tilings 
of a hexagon with side lengths a, b + to, c, a + to, b, c + to, where an equilateral tri- 
angle of side length m has been removed from the center. We give closed formulas 
for the plain enumeration and for a certain (— l)-enumeration of these lozenge tilings. 
In the case that a = b = c, we also provide closed formulas for certain weighted 
enumerations of those lozenge tilings that are cyclically symmetric. For m = 0, the 
latter formulas specialize to statements about weighted enumerations of cyclically 
symmetric plane partitions. One such specialization gives a proof of a conjecture of 
Stembridge on a certain weighted count of cyclically symmetric plane partitions. The 
tools employed in our proofs are nonstandard applications of the theory of nonin- 
tcrsccting lattice paths and determinant evaluations. In particular, we evaluate the 
determinants deto^jjXn-i + ( m j- )), where uj is any 6th root of unity. These 
determinant evaluations are variations of a famous result due to Andrews (Invent. 
Math. 53 (1979), 193-225), which corresponds to u = 1. 



Let a, b and c be positive integers, and consider a hexagon with side lengths a, 6, c, a, b, 
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1 Here and in the following, by a lozenge we mean a rhombus with side lengths 1 and angles of 60° 
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where H(n) stands for the "hyperfactorial" YYk=o ^- This follows from a bijection (cf. 
0) between such lozenge tilings and plane partitions contained in an a x b x c box, and 
from MacMahon's enumeration p5| , Sec. 429, q — > 1; proof in Sec. 494] of the latter. 

In jj^] (see also |53|]), Propp posed several problems regarding "incomplete" hexagons. 
For example, Problem 2 in |32j (and p3|| ) asks for the number of lozenge tilings of a 
hexagon with side lengths n, n+1, n, n+1, n, n+1 with the central unit triangle removed. 
This problem was solved in |^, Theorem 1] , [JH], Theorem 20] and [31 , Theorem 1] (the 
most general result, for a hexagon with side lengths a, b + l,c, a + 1,6, c + 1, being 
contained in [|3TJ] ) . In ||, the first author considers the case when a larger triangle (in 



fact, possibly several) is removed. However, in contrast to [31|, the results in || assume 



that the hexagon has a reflective symmetry, i.e., that b = c. 

Continuing this line of research, in this paper we address the general case, when no 
symmetry axis is required. We consider hexagons of sides a,b + m,c,a + m,b,c + m 
(in clockwise order) with an equilateral triangle of side m removed from the center (see 
Figures [l] and |2| for examples). We call this triangle the core, and the leftover region, 
denoted C ai b iC (m), a cored hexagon. 

To define C a ,b,cijn) precisely, we need to specify what position of the core is the 
"central" one. Let s be a side of the core, and let u and v be the sides of the hexagon 
parallel to it. The most natural definition (and the one that we are going to adopt) 
would require that the distance between s and u is the same as the distance between v 
and the vertex of the core opposite s, for all three choices of s. 

However, since the sides of the core have to be along lines of the underlying triangular 
lattice, it is easy to see that this can be achieved only if a, b and c have the same parity 
(Figure [I] illustrates such a case); in that case, we define this to be the position of the 
core. On the other hand, if for instance a has parity different from that of b and c, the 
triangle satisfying the above requirements would only have one side along a lattice line, 
while each of the remaining two extends midway between two consecutive lattice lines 
(this can be seen from Figure §). To resolve this, we translate this central triangle half 
a unit towards the side of the hexagon of length b, in a direction parallel to the side of 
length a, and define this to be the position of the core in this case. 

Note that, when translating the central triangle, there is no "natural" reason to do it 
in the sense we chose: we could have just as well chosen the opposite sense, obtaining an 
alternative (and not less central) definition of the core. However, it is easy to see that 
the alternative definition does not lead to new regions: it generates the same region that 
we obtain by swapping b and c in our definition. (In fact, this ambiguity in choosing 
the center will be used effectively in Section 0, see Theorem [29| and the paragraph 
preceding it.) 

Our main results, given in Theorems [I] and ^ below, provide explicit formulas for 
the total number of lozenge tilings of such a cored hexagon (see Figures |3] and |8|.a for 
examples of such tilings). Remarkably, the results can be expressed in closed form, more 
precisely, as quotients of products of hyperfactorials (completely analogous to formula 
Ql.lD ), thus providing an infinite family of enumerations which contains MacMahon's 
"box formula" ( II . 1| ) as a special case. For the statement of the theorems, it is convenient 
to extend the definition of hyperfactorials to half-integers (i.e., odd integers divided 
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H(n) :-- 



by 2): 

rifc=o r(fe + 1) for n an integer, 
rifc=o F(A; + |) for n a half-integer. 

Now we are able to state our theorems. The first result addresses the case that a, 
b and c have the same parity. Let L(R) stand for the number of lozenge tilings of the 
region R. 

Theorem 1. Let a,b,c,m be nonnegative integers, a,b,c having the same parity. The 
number of lozenge tilings of a hexagon with sides a,b + m,c,a + m,b,c + m, with an 
equilateral triangle of side m removed from its center (see Figure [| for an example) is 
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L(Ca,b,c{m)) = 

H(a + m)H(6 + m)H(c + m)H(a + 6 + c + m) H(m + \ K( m + \*^\) 
H(a + 6 + m) H(a + c + mj H(6 + c + m) H(^ + m) H(^±£ + m) H(Si + m ) 

H( ff 1 ) H( [I] ) H( [f 1 ) H( [f J ) H( L| J ) H( LfJ ) 



x 



H(f + [|1 ) H(f + [fl ) H(f + \l 1 ) H(f + LfJ ) H(f + L|J ) H(f + LfJ ) 

X H(f + f^Pf) H(f + L^J) H(^) H(s±£) H(^) ' (L2) 

Clearly, formula ( |1.2|) reduces to ( [L.l|) for m = (as it should). The special case 
m = 1 has been obtained earlier in |H| Theorem 1]. 

The corresponding result for the case when a, b and c do not have the same parity 
reads as follows. 



Theorem 2. Let a,b,c,m be nonnegative integers, with a of parity different from the 
parity of b and c. The number of lozenge tilings of a hexagon with sides a,b + m,c,a + 
m,b,c + m, with the "central" (in the sense described above) triangle of side m removed 
(see Figure ^for an example) is given by 



L(C atb}C (m)) = 

H (a + m) H (b + m) H (c + m) H (a + b + c + m) H (m + f 2 ^] ) H (m + [^^l ) 
R(a + b + m)R(a + c + m)R(b + c + m)R ([^±£j + m) H + m) H (pp] + m) 

H(f) 2 H([fl)H([|l)H([ f l)H([fj)H([|j)H([|j) 
H (f + [fl) H (f + [fl)H (f + [|l)H(f + LfJ) H (f + LfJ) H (f + LfJ) 
H(m+f)H([^j+f)H([^j+f)H([^]+f)H(^ + f) 2 

H(f +[^l)H(f+[^j)H(L^J)H([^l)H(^) • l - J 

Again, formula (|1.2|) reduces to ( |1 . 1|) for m = 0. The special case m = 1 has been 
obtained earlier in |JT], Theorem 4]. 

Given the explicit results in Theorems [l] and [|, it is routine to determine, using 
the Euler-MacLaurin summation formula, the asymptotic behavior of the number of 
lozenge tilings of a cored hexagon. For instance, when a, b and c have the same parity 
we obtain the following result. 



Corollary 3. Let a,b 7 c,m,n be nonnegative integers, a,b,c having the same parity. 
The number of lozenge tilings of a hexagon with sides an, (b + m)n,cn, (a + m)n, bn, (c + 
m)n, with an equilateral triangle of side mn removed from its center, is asymptotically 
given by 

L{C anM)Cn (mn)) ~ e kn \ n -> oo, 
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where 

k = log(a + m) + log(6 + m) + log(c + m) 

+ (a+fc+ 2 c+m)2 log(a + b + c + m) + 2(m + ^±§±£) 2 log(m + s±f±s) 

+ 2(f) 2 log(|) + 2(f) 2 log(f ) + 2(f) 2 log(|) + (f ) 2 log(m) 
— f| (a + b + m) 2 log(a + 5 + m) + |(a + c + m) 2 log (a + c + m) 
+ | (ft + c + m) 2 log(6 + c + m) 

+ (0±6)2 log(a + 6 ) + (a±c )2 ]og(a + c) + ( 6±c )2 1()g(6 + c) j 

+ (m 2 + a 2 + 6 2 + c 2 + 3m(a + b+c) + a& + be + ca) log 2. (1.4) 

In addition to plain counts, (— l)-enumerations of plane partitions, i.e., enumerations 
where plane partitions are given a weight of 1 or —1, according to certain rules, have 
been found to possess remarkable properties (see |]38], |59|). Motivated in part by a con- 
jectured (— l)-enumeration on cyclically symmetric plane partitions due to Stembridge 
|40[ |, in Section |2| we consider a (— l)-enumeration of the lozenge tilings of Theorems [IJ 
and 0. The corresponding results are given in Theorems |] and [S[ 

In Section |^, we restrict our attention to cyclically symmetric lozenge tilings (i.e., 
tilings invariant under rotation by 120°) of cored hexagons. Clearly, this makes sense 
only if a = b = c, i.e., for cored hexagons of the form C a ^ a {m). The plain enumer- 
ation of such cyclically symmetric lozenge tilings had already been considered in || 
Theorem 3.2 and Corollary 3.3]. We restate the result here as Theorem |6]. We provide 
several additional results. Theorem |7| concerns the (— l)-enumeration of such cyclically 
symmetric lozenge tilings and some additional weighted enumerations of them, where 
each lozenge tiling is weighted by some 6th root of unity, according to a certain rule (see 
the paragraph before Theorem [j] for the precise definition). In the special case m = 
we obtain results about weighted enumerations of cyclically symmetric plane partitions 
(see Corollary 13). A particular case of Corollary 13 proves a conjecture of Stembridge 



4*0| , Case 9 on p. 6] about a certain (— l)-enumeration of cyclically symmetric plane 
partitions. (The first proof of this conjecture, by totally different means, is due to 
Kuperberg |22|, last displayed equation on p. 27].) Our results also allow us to prove 
another conjecture on (— l)-enumeration of cyclically symmetric plane partitions due to 
Stembridge [HO, Case 10 on p. 7]. In fact, we again prove a more general result, namely 
a result on cyclically symmetric lozenge tilings (see Theorem |^). 

The remaining sections, Sections ^TT|, are devoted to the proofs of these results. 
For the proofs of Theorems [TJ-[5], the enumeration results for lozenge tilings without 
symmetry, we proceed as follows. First, we identify tilings with certain families of 
nonintersecting lattice paths (see Section ^). Then, a nonstandard application of the 
main theorem on nonintersecting lattice paths |23| , Lemma 1], |13| , Theorem 1] (restated 



here in Lemma [14]) provides a determinant for the weighted count of lozenge tilings (see 
Q5.4D , respectively Q5.5Q ). To be precise, the determinant gives the correct weighted 
count either only for even m (m being the side of the core) or only for odd m, depending 
on whether we are considering plain enumeration or (— l)-enumeration. To cover the 
other case as well, we prove that the weighted count of lozenge tilings that we are 
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interested in is polynomial in m, so that it suffices to determine this number only for 
one of the two possibilities, either for even m or for odd m. This is in turn achieved by 
evaluating the aforementioned determinant (see Lemmas |IT|-p4|) . 



The results on weighted enumerations of cyclically symmetric lozenge tilings in Sec- 
tion |3| can be obtained in a similar way. We phrase the problem in terms of nonintersect- 
ing lattice paths, and thus find determinants for these enumerations. The determinants 
have the form 

„^4^-+( ro+ r))' <L5) 

where u is any 6th root of unity. These determinants are remarkable. The case uo = 1 
occured first in the work of Andrews on plane partitions. He evaluated the determinant 
( p..5[ ) in that case [0, Theorem 8] (restated here as Theorem |irj) in order to prove the 
"weak Macdonald conjecture" on counting cyclically symmetric plane partitions. It had 
already been observed in |6|, Sec. 3] that Andrews' evaluation of ( p..5|) with u> — 1 gives 
the number of cyclically symmetric lozenge tilings of the cored hexagon C a ^ a {m). We 
prove our weighted enumerations of these lozenge tilings by evaluating the determinant 
( p..5|) when u is any 6th root of unity (see Theorems pTHl3|) . 

Our paper is structured as follows. In Section || we give the precise definition of our 
(— l)-enumeration of lozenge tilings, and we state the corresponding results (see The- 
orems |] and |^). In Section |3| we define precisely our unusual weightings of cyclically 
symmetric lozenge tilings. Theorems |6] and [7|, Corollary |8] and Theorem |9| state the 
corresponding results. The subsequent section, Section |], gives the proofs of our enu- 
meration results in Theorems [1|^9|, leaving out, however, several details. These details 
are then worked out in later sections. First of all, in Section |5|, it is explained how 
lozenge tilings correspond, in a one-to-one fashion, to families of nonintersecting lattice 
paths. We then employ the result of Lemma [b| to obtain, at least for every other value 



of m, a determinant for the weighted count of lozenge tilings that we are interested in 
(see Lemmas [13] and [TO). It is then argued in Section that this number is in fact 



polynomial in m, so that the evaluation of the determinant in Lemma [TJ| respectively 
Lemma |TB|, suffices. The precise form of the evaluation of the determinant in Lemma [TJ] 
(again, a case-by-case analysis is necessary, depending on the parity of a) is stated and 
proved in Section |7| (see Lemmas |i7H20[) , while the precise form of the evaluation of the 



determinant in Lemma is stated and proved in Section |8| (see Lemmas f2~T|-f2"4]) . Fi- 
nally, in Section |] we prove the determinant evaluation of Theorem [11 , in Section [10| the 



one in Theorem [TJ| and in Section [IT] the one in Theorem |13|. We conclude the article 
with some comments concerning connections of this work with multiple hypergeometric 
series and some open problems. These are the subject of Section O. 



2. (-l)-ENUMERATIONS OF LOZENGE TILINGS OF CORED HEXAGONS 

In this section we enumerate lozenge tilings of a cored hexagon with respect to a 
certain weight that assigns to each lozenge tiling the value 1 or —1. More precisely, 
fix a lozenge tiling T of the cored hexagon C a ,b ,c( m ) ( see Figures [lj and || for examples 
of such regions, and Figure |3| for an example of a tiling; at this point, the thickness of 
edges is without significance). Consider the side of the core which is parallel to the sides 
of the hexagon of lengths a and a + m (in the figure this is the bottommost side of the 
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c + m 




FIGURE 3. A lozenge tiling of a hexagon with sides a 
removed triangle of side length m = 2. 



5, b = 3, c = 1 and 



core). Extend this side of the triangle to the right. Let n(T) be the number of edges of 
lozenges of the tiling T contained in the extended side (in Figure |3] there are two such 
edges, marked as thick segments). The statistic n(T) becomes most transparent in the 
lattice path interpretation of lozenge tilings that is going to be explained in Section |5], 
as it counts exactly the number of paths which pass the core on the right. Furthermore, 
we shall see in Section ^ that in the plane partitions case, i.e., in the case m = (when 
the core shrinks to a point), the statistic n(T) has a very natural meaning as well (see 
the remarks after Theorem [?D . 

In the (—1) -enumeration, which is the subject of the following two theorems, each 
lozenge tiling T is weighted by (— l) n{ ~ T \ Let L~ 1 (R) be the weighted count of lozenge 
tilings of region R under the above weight. 

Theorem 4. Let a, b, c, m be nonnegative integers. If all of a, b and c are even, then 
the weighted count 1) ; summed over all lozenge tilings T of a hexagon with 
sides a,b + m,c,a + m,b,c + m, with an equilateral triangle of side length m removed 
from its center (see Figure [/[) is given by 

L-\C aAc {m)) = 



■1) 



a/2 H(a + m) H(6 + m) H(c + m) H(a + b + c + m) 
H(a + b + m) H(o + c + m) H(6 + c + m) 



x 



x 



H(f) 2 H(f) 2 H(§; 



H 



m— 1 1 



H 



• m+l ' 



H(§ + *=i; 

•pj^ a+b+m— 1 ' 



H(| + 



m— 1 ' 



H(| 



H( 



a+fe+m+l ' 



m— 1 n 
-2 1 2 > 
tj/ a+c+m— 1 ' 



H(f + 



m+l ' 



H(| 



m+l ' 



2 / iJ -V2 1 2 ■ 
tj/ a+c+m+1 \ tj/ b+c+m— 1 ' 



H(f + 



m+l ' 



H 



6+c+m+l " 



H(2±2) H(2±£) H(^) H(2±2 + m) H(^ + m) H(*±£ + m 



H(2i|+e+m) 2 



H 



' a+b+c 



+ 



m+l ' 



(2.1) 



Fora,b,c all odd, the (— 1)- enumeration equals zero. 
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FIGURE 4. A cyclically symmetric lozenge tiling of a hexagon with sides 3, 
5, 3, 5, 3, 5 and core of size m = 2. 



The analogous theorem for the case when a has a parity different from the parity of 
b and c reads as follows. 

Theorem 5. Let a, b, c, m be nonnegative integers, a of parity different from the parity 
of b and c. The weighted count ^ (—l) n( - T \ summed over all lozenge tilings T of a 
hexagon with sides a,b + m,c,a + m,b,c + m, with an equilateral triangle of side length 
m removed that is "central" in the sense that was described in the Introduction (see 
Figure equals 

L-\C a , c {m)) = 



_-Q [a/21 



X 



H(a + m) H(6 + m) H(c + m) H(a + b + c + m) 
H(a + b + m) H(a + c + m) H(& + c + m) 
H([2±|+£j +m)H([2±|±2] +m) 



H (a+!+l 



m) H(^±pl + m ) H(^ + m) 



x 



H( LU ) H( [f 1 ) H( LDJ H( [|1 ) H( LfJ ) H( \l 1 ) H( V) H ( 



m+l ' 



rra+1 



X 



2 ■ hrl) H ( 2 

a+b+m \ 2 jj^ a+c+m^2 fc+c+m— 1 ^ 6+c+m+l ' 



m— 1 



L^J)H(^1+M)H( 



m—l 



L^J) 



0+6-1' 



H(^±f±i) H(^±£) H 



m— 1 



J)H( 



m+l 



(2.2) 



3. Enumeration of cyclically symmetric lozenge tilings 

In this section we enumerate cyclically symmetric lozenge tilings of the cored hexagon 
C a (m) := C a ^ )a {m) with respect to certain weights. By a cyclically symmetric lozenge 
tiling we mean a lozenge tiling which is invariant under rotation by 120°. See Figure f|for 
an example. (At this point, all shadings, thick and dotted lines should be ignored.) The 
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unweighted enumeration of these lozenge tilings was given earlier in [0, Theorem 3.2 and 
Corollary 3.3]. We restate the result below. Let L C (R) denote the number of cyclically 
symmetric lozenge tilings of region R. 

Theorem 6. Let a be a nonnegative integer. The number L c (C a (m)) of cyclically sym- 
metric lozenge tilings of a hexagon with side lengths a, a + m,a,a + m,a,a + m, with 
an equilateral triangle of side length m removed from the center, equals the right-hand 
side in fO). □ 



Let us now associate certain weights to each such lozenge tiling T. These weights 
depend again on the number n{T) of edges of lozenges of the tiling T which are incident 
to the extension to the right of the bottommost side of the core. (Since we are now 
dealing with cyclically symmetric tilings, it does, in fact, not matter which side is 
considered, and the weighted count is not even affected by the choice of direction.) In 
the following three theorems, each lozenge tiling T is assigned the weight uj n<yT \ where 
uj is some fixed 6th root of unity. Denote by L^(R) the corresponding weighted count 
of cyclically symmetric lozenge tilings of region R. 

Theorem 7. Let a > and m > be integers. Then the weighted count L^(C a (m)) : = 
^uj n ^ T \ summed over all cyclically symmetric lozenge tilings T of a hexagon with side 
lengths a,a + m,a,a-\-m,a,a + m, with an equilateral triangle of side length m removed 
from the center, equals the right-hand side in (|3~3| ) if uj = — 1, it equals the right-hand 
side in ( |3.4|) if uo is a primitive third root of unity, and it equals the right-hand side in 
( p.5| ) if uo is a primitive sixth root of unity. 

If we specialize these results to m = 0, i.e., to the case where there exists no core, we 
obtain enumeration results for cyclically symmetric plane partitions. Before we state 
these, let us briefly recall the relevant notions from plane partition theory (cf. e.g. 



36| or IH, Sec. 1]). There are (at least) three possible equivalent ways to define plane 
partitions. Out of the three possibilities, in this paper, we choose to define a plane 
partition tt as a subset of the three-dimensional integer lattice (where Z + denotes 
the set of positive integers), with the property that if (ii, ji, k±) is an element of tt, then 
all points (i2,j2,k 2 ) with 1 < i 2 < i%, 1 < J2 < ji, 1 < k 2 < ki also belong to tt. (In 
the language of partially ordered sets, tt is an order ideal of Zq_.) A plane partition tt is 
called cyclically symmetric if for every (2, j, k) in tt the point (j, k, i) which results by a 
cyclic permutation of coordinates is in tt as well. 

Often, a plane partition is viewed as the corresponding pile of unit cubes which 
results when replacing each point (i, j, k) of the plane partition by the unit cube with 
center (i, j, k). A three-dimensional picture of a plane partition, viewed as pile of unit 
cubes, is shown in Figure [5] (in fact, this example is cyclically symmetric). As we 
already mentioned in the Introduction, plane partitions contained in an a x h x c box 
(i.e., plane partitions tt with the property that every (i,j,k) G tt satisfies 1 < i < a, 
1 < j < b, 1 < k < c) are in bijection with lozenge tilings of a hexagon with side 
lengths a, b, c, a, b, c (see 0). This bijection can be visualised easily on the example in 
Figure [5[ Clearly, under this bijection, cyclically symmetric plane partitions contained 
in an a x a x a box correspond to cyclically symmetric lozenge tilings of a hexagon 
with all sides of length a. Thus, Theorem [7] with m = yields results about certain 
weighted counts of cyclically symmetric plane partitions. We just have to figure out 
how the weights uj n ^ for lozenge tilings T translate to the plane partition language. 




FIGURE 5. A cyclically symmetric plane partition. 



Let ttt be the plane partition that corresponds to the lozenge tiling T under this 
bijection. Denote by thi^t) the number of elements of the form in ttt- Then 

there are precisely thi^t) un h cubes on the main diagonal of the pile of unit cubes 
representing n?- Let v be the vertex farthest from the origin of the last such unit cube 
(in the planar rendering of ht — for our example, Figure [| — v is the center of the 
hexagon). A ray through v approaching orthogonally any of the coordinate planes will 
cut through precisely mi(7Tx) layers of unit thickness. Since each such cut corresponds 
to a lozenge side contained in the ray, we see that tux^t) is precisely the statistic n(T). 
We therefore obtain the following corollary of Theorem [7|. 

Corollary 8. Let a be a nonnegative integer. Then the weighted count ^u 1711 ^, 
summed over all cyclically symmetric plane partitions 7r contained in an a x a x a 
box, equals the right-hand side in ( |3.3| ) with m = if u = —1, it equals the right- 
hand side in Q3.4Q with m = if u is a primitive third root of unity, and it equals the 
right-hand side in (|3.5|) with m = if u is a primitive sixth root of unity. □ 



Weighted enumerations of this sort have been considered earlier. In fact, the result 
for uj = — 1 of Corollary || had been conjectured by Stembridge |4(| Case 9 on p. 6], 
and proved for the first time by Kuperberg |22| , last displayed equation on p. 27]. 
Thus, the (— l)-result of Theorem [7|is a generalization of Kuperberg's result. There are 
many more conjectures on (— l)-enumerations of cyclically symmetric plane partitions 
in [Q. One of these, the Conjecture on p. 7 of |40], Case 10], asks for the weighted 



count l) m6 ( 7r ) of cyclically symmetric plane partitions in which the statistic m^ii) 
is defined as the number of orbits (under cyclic rotation) {(i,j,k), (j,k,i), (k,i,j)} of 
elements of 7r with coordinates that are not all equal. 

We prove this conjecture of Stembridge in Theorem |9| below. In fact, in Theorem |9] we 
prove a result for cyclically symmetric lozenge tilings of cored hexagons. In this result, 
a cyclically symmetric lozenge tiling T is given a weig ht (-l) n6 ( T ), with the statistic 
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n§(T) to be described below. It is defined in a way so that in the case when there is 
no core present (i.e., m — 0) it reduces to itiq^tit), where again ttt denotes the plane 
partition corresponding to T. 

Let T be a fixed cyclically symmetric lozenge tiling of the cored hexagon C a (m) (see 
Figure || for an example with a = 3 and m = 2; at this point, all thick lines and shadings 
should be ignored). We consider the horizontal lozenges which are at least partially 
contained in the top-right fundamental region. (In Figure || the top-right fundamental 
region is framed. The horizontal lozenges which are at least partially contained in that 
region are the grey and black lozenges.) The statistic n§(T) is by definition the sum 
of the vertical distances between these horizontal lozenges and the lower border of the 
fundamental region. (Thus, for the lozenge tiling To in Figure |G] we have, considering 
the horizontal lozenges in the order from left to right, n§(To) = 2 + 1 + + + = 3.) 

Suppose now that m = 0, and view the tiling T as a plane partition n?. The 
fundamental region of T used in our definition of the statistic Uq corresponds to a 
fundamental region of it?* with the main diagonal removed. Since the distances we add 
up in our definition of n§(T) are precisely the heights of the vertical columns of unit 
cubes in this fundamental region, we obtain that n§(T) is equal to the number of unit 
cubes contained in it, which is clearly just the number of orbits of cubes off the main 
diagonal. This verifies our claim that n§(T) = itlq{tit)- 

The weight which is assigned to a tiling T in the theorem below is (— l) n6 ( T ). An 
equivalent way to define this weight is to say that it is the product of the weights of 
all lozenges which are, at least partially, contained in the top-right fundamental region, 
where the weight of a horizontal lozenge with odd distance from the lower border of the 
region is —1, the weight of all other lozenges being 1. (In Figure || the black lozenge has 
weight —1, all other lozenges have weight 1.) Yet another way to obtain this weight is 
through the perfect matchings point of view of lozenge tilings, elaborated for example 
in [pi], |2"2fl. In this setup, the cyclically symmetric lozenge tilings that we consider here 
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correspond bijectively to perfect matchings in a certain hexagonal graph (basically, the 
dual graph of a fundamental region of the cored hexagon). Assignment of weights to 
the edges of this graph so that each face has "curvature" —1 (see |22|, Sec. II]) generates 
again (up to a multiplicative constant) the above weight for lozenge tilings. 

Denote by L G _1 (i?) (where the index letter stands for "orbits") the weighted count of 
lozenge tilings of region R under the above-defined weight. 



Theorem 9. Let a and m be nonnegative integers. Let R±(a, m) denote the right-hand 
side of ( |3.2| ), let R 2 (a,m) denote the right-hand side of ( 13.31 ), and let R 3 (a,m) denote 
the right-hand side of ( p.5|) . Then the weighted count XX ~ l) n ^ T \ summed over all 
cyclically symmetric lozenge tilings T of a hexagon with side lengths a,a + m,a,a + 
m,a,a + m with an equilateral triangle of side length m removed from the center, is 
given by 

if a is even and m is even, 

if a is odd and m is even, 
J (3.1) 
if a is even and m is odd, 

if a is odd and m is odd. 

As we show in Section f|, all the above results in the current section follow from 
evaluations of the determinant ( |1.5|) for u equal to 1, to —1, to a primitive third 
root of unity, and to a primitive sixth root of unity, respectively. The corresponding 
evaluations read as follows, the evaluation for u — 1, given in Theorem [10] below, being 
due to Andrews 0, Theorem 8]. 

Theorem 10. For any nonnegative integer a, 



ri^(f,f)i 2 

fli^S- 1)^(2=1,22 + 1) 
d (a m— 1\ d (a m+1 \ 

d I o+l m— 1\d (a— 1 m+l\ 
l^UT' -Q-J-rM-o-, —2~) 



det 5n + 

0<i,j<a-l \ \ J 

a-2 



m + % + j 



2 ra/*l T\(-+\i/2]+l) 

i\ V2 1/1 ; L(i+ 3)/4j 

na/2 lm _|_ 3a _ [3i1 , 3\ (m , 3a _ V3il , 3\ 

i=lV2~ t ~2 \ 2 \ ^~ 2> ri/21-1 V2~ t ~2 I 2 I "l" 2 / U/2] ., 

x 77-- it a is even, 

n:5" 1 (2^-l)!!(2z + l)!! 

2 ra/2i n(y+R/2l+l 



i=l 

T-r(a-l)/2 fm , 3a _ f3iziil _L 1 "| /m 1 3a _ |~3iT\ 

lli=l I 2 "T" 2 2 I "T" L J r(i-l)/2l ^2^2 I 2 I / \i/2] ., . 



r(*+3)/41 



n£T 1)/2 (2^-i)!! 



(3.2) 



where (a)k is the standard notation for shifted factorials, (a)& := a(a + l) • ■ • (a+/c — 1), 
> 1, and (a)o := 1. □ 
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Theorem 11. For nonnegative integers a, 

det (-6, ]+ ( m + i + r 

0<i,j<a-l \ \ J 

if a is odd, 

_iW2 rra/2-1 i|a( f +0!2 (f+3i +l)|'( m+ 3i+l)!^ , (3.3) 

^ 1 li=0 (2i)!(2i+l)!(f +2i)! 2 (f +2j+l)! 2 (m+2i)!(m+2i+l)!' 1 

The proof of this theorem is given in Section 
Theorem 12. Let uj be a primitive third root of unity. Then 
'm + i + j\\ (l + u) a 2^ 



det uSa + 



o<i,i<a-i V 13 V J / / nl=i 2J (2i - 1)H nliT 1)/2J (2i - 1)" 
x J] (f + 3i + 1) [(a _ 4j)/2j (f + 3. + 3) L(a „ 4i _ 3)/2J 

" (f + a ~ 1 + I) L(a-4t-l)/2j (f + ~ * _ I) L(a-4t-2)/2j ' ^ 3 ' 4 ) 

where, in abuse of notation, by [a\ we mean the usual floor function if a > 0, however, 
if a < then [a\ must be read as 0, so that the product over i > is indeed a finite 
product. 

The proof of this theorem is given in Section 
Theorem 13. Let u be a primitive sixth root of unity. Then 

'm + i+j\\_ (i + ^r(i) La/2J 



det u>5. 



o<y<a-i v 13 V J ) ) nti 2J (2i - 1)!! ll=T 1)/2J - 1)H 

X II (f + 3i + I) L(o-4i-l)/2j (f + 3l + I) L(o-4i-2)/2J 
i>0 

• (y + a - z) L(o-4i)/2j ( Y + « — i) L(a-4i-3)/2j , (3.5) 

where again, in abuse of notation, by [_a\ we mean the usual floor function if a > 0, 
however, if a < then [aj must be read as 0, so that the product over i > is indeed 
a finite product. 

The proof of this theorem is given in Section |TT|. 

4. Outline of the proofs of Theorems [T]-[9] 

In this section, we give outlines of the proofs of our enumeration results stated in 
the Introduction and in Sections @ and |||. We fill in the details of these proofs in later 
sections. 

Proof of Theorem [J. There is a standard bijection between lozenge tilings and families 
of nonintersecting lattice paths. This bijection is explained in Section [5] (see in particular 
Figure §). Thus, the problem of enumerating lozenge tilings is converted to the problem 
of counting certain families of nonintersecting lattice paths. By the Lindstrom-Gessel- 
Viennot theorem (stated in Lemma 0), the number of such families of paths can be 
expressed as a determinant (see Lemma |15D. Thus, in principle, we would be done once 
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we evaluate this determinant, given in (|5.4j). However, Lemma [15] applies only if the 



size m of the core is even. We show, in Section || that it suffices to address this case, 
by proving that the number of lozenge tilings that we are interested in is a polynomial 
in m. The evaluation of the determinant (|5.4p for even m is carried out in Section [7] 
(see (|7.1[ ) and Lemmas [T7| and |TSJ) . □ 



Proof of Theorem [|. The first steps are identical with those in the preceding proof: 
the lozenge tilings are converted into nonintersecting lattice paths, in the way that 
is described in Section Therefore, Lemma TA yields a determinant for the (— 1)- 



enumeration that we are interested in. Unlike in the previous proof, this provides a 
determinant for our weighted count only if the size m of the core is odd (see Lemma |15|). 
Again, the considerations in Section [] show that this number is a polynomial in m, so 
it suffices to evaluate the determinant (5.4) for odd m. This is done in Section [5] (see 



J]) and Lemmas [19] and ]2(|) . □ 



Proof of Theorem [|. Again, we use the strategy from the proof of Theorem |IJ We 
convert the lozenge tilings into families of nonintersecting lattice paths as described 
in Section |5|. The starting and ending points are slightly different from the ones used 



before. They are given in (|5.2| ). Lemma [14] yields a determinant for the number we 



are interested in for even m (see Lemma |16|). The considerations of Section |6] still 
apply, so the number of lozenge tilings is a polynomial in m and it suffices to evaluate 



the determinant (|5.5|) for even m. This is accomplished in Section || (see ( |S.1|) and 



Lemmas Ell and E2[) . □ 



Proof of Theorem [|. We proceed analogously to the proof of Theorem 0. The lozenge 
tilings are converted into nonintersecting lattice paths, in the way that is described in 
Section [5|. Therefore, Lemma [14] yields a determinant for the (— l)-enumeration in the 
case of odd m (see Lemma 0). Again, the considerations in Section |6| show that this 
number is a polynomial in m, so that it suffices to evaluate the determinant (|5.5|) for 



odd m. This is worked out in Section R] (see (|8.1|) and Lemmas |23] and [24]). □ 



Proof of Theorem [?[. We follow the arguments of the proof of Theorem |||, as given in 
|6|, Lemma 3.1]. Suppose we are given a cyclically symmetric lozenge tiling T of our 
cored hexagon C a {m). It is completely determined by its restriction to a fundamental 
region, the lower-left fundamental region, say. (In the example in Figure pjL the lower- 
left fundamental region is framed.) Some of the lozenges are cut in two by the borders of 
the fundamental region. (In Figure |] these are the shaded lozenges.) We draw lattice 
paths which connect these "cut" lozenges, by "following" along the other lozenges, 
as is indicated in Figure |3] by the dashed lines. To be precise, in each lozenge in 
the interior of the fundamental region, we connect the midpoints of the sides that 
run up-diagonal, in case the lozenge possesses such sides. Clearly, these paths are 
nonintersecting, by which we mean that no two paths have a common vertex. Since 
they determine completely the cyclically symmetric lozenge tiling, we may as well count 
all these families of nonintersecting lattice paths, with respect to the corresponding 
weight. In fact, as is easy to see, because of the cyclic symmetry, the statistic n{T) 
is exactly equal to a minus the number of paths. If we fix the "cut" lozenges, say in 
positions z'x, i 2 , ■ ■ ■ , ik (counted from inside out, beginning with 0; thus, in Figure f|, the 



'cut" lozenges have positions and 2), then, according to Lemma [14|, the number of 
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Figure 7. The orthogonal path corresponding to Figure |6| 



families of nonintersecting lattice paths connecting the fixed "cut" lozenges is given by 
the corresponding Lindstrom-Gessel-Viennot determinant (the left-hand side of ( |5.3|) ). 

This determinant turns out to be the minor of ( ( m+J+J ) \ consisting of rows 

V 3 / 0<i,j<a-l 

and columns with indices ii,i 2 , . . . ,ik- This number must be multiplied by the common 
weight uj a ~ k of these families of nonintersecting lattice paths. Therefore, in order to 
obtain the total weighted count that we are interested in, we have to sum all these 
quantities, i.e., take the sum of 



({11,12, • • • , ifc)-principal minor of ({ m+ * +3 )} 



0<i,j<a-l 



X U a k 



over all k = 0,1, ... ,a and < i\ < i 2 < ■ ■ ■ < ik < a — 1. Clearly, this sum is 

exactly equal to deto<ij< a -i (uSy + ( m4 ? i+ "') j, which equals the left-hand side of ( |3.3| ) 

if a; = —1, the left-hand side of (|3.4j ) if uj is a primitive third root of unity, and the 
left-hand side of (|3.5|) if to a primitive sixth root of unity. The respective right-hand 
sides provide therefore the solution to our enumeration problem. □ 

Proof of Theorem || We adapt the arguments used in the proof of Theorem [j]. (Clearly, 
here we want to count the same objects, but with respect to a different weight.) So, 
again, we draw paths that connect the lozenges which are cut in two by the borders of 
the fundamental region. This time, we choose the top-right region as the fundamental 
region. Figure |6] shows an example. There, the top-right fundamental region is framed. 
As in Figure ^, paths are indicated by dashed lines. (In the example in Figure |] there 
is just one path.) If we slightly distort the underlying lattice, we get orthogonal paths 
with positive horizontal and negative vertical steps. Figure [?| shows the orthogonal 
path corresponding to the path in Figure ^. The manner in which we have chosen the 
coordinate system ensures that possible starting points of paths are the points (0, j), 
< j < a — 1, and possible ending points are the points (ra + i, 0), < % < a — 1. 

Now, as before, we fix the positions of the "cut" lozenges. Then a weighted version of 
the Lindstrom-Gessel-Viennot theorem (see p3| , Lemma 1] or [|T^, Cor. 2]) can be used 
to express the weighted count of the corresponding families of nonintersecting lattice 
paths in form of a determinant. In fact, this weighted version just says that Lemma 
remains true when the number V{A — > E) of paths from A to E is replaced everywhere 
by the weighted count ^2 P w{P) of all paths P from A to E, where w is some weight 
function on the edges of the square lattice and the weight w{P) of a path is the product 
of the weights of its steps. Thus, if we repeat the subsequent arguments in the proof of 
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Theorem [7], then we obtain the determinant 



det 

0<jj<a- 



U 3 + J2 

> 13. rn n^ / 



(4.1) 



P:(0,i)-»(m+i,0) 



for the weighted count of our families of nonintersecting lattice paths. 

We now choose the weight function w so that the weight of the family of noninter- 
secting lattice paths corresponding to a tiling T is equal to (—l) na( - T \ To do this, it will 
be convenient to stick on an extra initial horizontal step at the beginning of each path, 
so that now it starts on the line x = —1. Weight the vertical steps on this line by 0, 
all the remaining vertical steps by 1, and weight horizontal steps at height j by (— 1) J '. 
Since the height of a horizontal step is equal to the distance of the corresponding hori- 
zontal lozenge to our reference line in the tiling, the weight of a family (Pi, P 2 , . . .) of 
nonintersecting lattice paths is equal to (— l) n6 ( T ), where T is the corresponding tiling. 
On the other hand, it is clearly equal to (— \} A ( p i)+ A i p 2)+ w here A(P) denotes the 
area between a path P and the 

To find an expression for the entries of the Lindstrom-Gessel-Viennot matrix we use 
the well-known fact (see |37], Prop. 1.3.19]) that the weighted count Yl Q A ^ P \ summed 
over all lattice paths P from (0, c) to (d, 0), is equal to [ C c d ]g> where [1] q is the standard 
g-binomial coefficient, 



[l-q n )(l- t 



Thus, the determinant 



(i_ g *0(i_ g *-i)...(i_<?) 

7TT) becomes (see also |40], Lemma 4]) 



det 



Sa + 



0<i,j<a-l 

From the g-binomial theorem (see |IJ (3.3.6)]), 



m + i + j 
j 



(4.2) 



[l + z)(l + qz)---(l + q 



n-l. 



k=0 



it is straightforward to extract that 







if n is even and k is odd, 
otherwise. 



(4.3) 



(\n/2\\ 
\\h/2\) 

We have to compute the determinant (|4.2|). Let us denote it by D . We have to 
distinguish between four cases, depending on the parities of m and a. 

First, let m be even. We reorder rows and columns simultaneously, so that the even- 
numbered rows and columns come before the odd-numbered, respectively. If a is even, 
then we obtain for Dq the block determinant 



det 



J(f) + J B(f,f; 
£(f,f) 



~r>(a m' 



where I(N) is the N x N identity matrix and B(N,m) is the N x N matrix 



0<i,j<N-l 



By a few simple manipulations, this determinant can be factored 
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into a product of two determinants, 

Dn-detf^ + ^i'f) ~ B &f< 

-det^ + ^f'f) - S (f'tA det ^ J (f) 
- aet { 5(f, f) /(f) ^^-SCf.f) /(f) 

_ det ^(f) + £(f,?) + £(f,f) 2 

" det ^ /(f) 

= det (I(i) + B(l,f) + B(lf) 2 ) 
= det ( W J(§) + £(§, f )) det (57/(f ) + fl(§, f )) , 
where is a primitive sixth root of unity, each of which can be computed by application 



of Theorem 12. The result is the first expression in (3.1). 

On the other hand, if a is odd, then analogous arguments yield 

D = det (J(s±i) + S(s±i, f ) + £( c )(2±±, f ) B (r) (f , f )) , (4.4) 

where £?( c )(^±i, y) is the (^) x (^y-) matrix which arises from B( 9 4^, y) by deleting 
its last column, while /r r )(^y-, y) is the (^y-) x (^y-) matrix which arises from B by 
deleting its last row. 
It is easy to check that 

J(^)+ J B(s±l, f )+ J B( c )(^±i, f ) B«(f, f ) = (/(2±1)+S) (J(2±l) + B(s±l, f - 1)) , 

where 5 is the (^y 3 -) x (^±l)-matrix with (z, -entry < i,j < (a - l)/2. 

(So the first column of B is zero). We expand det(/( s y-) + B) with respect to the first 
column and get det(/(^) + B(^, f + 1)). 

Therefore, in the case of even m and odd a, we have 

D = det (J(s±i) + B(s±i, f - 1)) det (1(2=*) + B(^, f + 1)) . 

Both determinants can be evaluated by means of Theorem |10|. The result is the second 
expression in ( |3.1| ). 

Now let m be odd. We proceed analogously. If a is even, then reordering rows and 
columns according to the parity of the indices gives 

D - det ( J ® + B & ^ ° 
u - aet ^ 5(f,^±i) 7(|) _ 5(2, 2^1; 

= det (/(f) + S(f , ^)) det (/(f) - 5(f, !2±!)) . 

The first determinant is evaluated by means of Theorem [10], while the second is evalu- 



ated by means of Theorem |TT[ The result is the third expression in (|3.1J ) . 
Finally, if a is odd we get 

fUS+l) + ™=1) 

1^0 — Ut - L I m+1 \ j/ a-l N _ m+1 

= det (J(2±i) + S(2±l, det (/(Syl) - fl(*yl, ^±i)) 



Again, the first determinant is evaluated by means of Theorem 10, while the second is 



evaluated by means of Theorem 11. The result is the fourth expression in (|3.1|). □ 
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5. Lozenge tilings, nonintersecting lattice paths, and determinants 

The purpose of this section is to derive determinants for the ordinary and (— 1)- 
enumeration of lozenge tilings of cored hexagons (see Lemmas [15| and [16]). We find 
these determinants by first translating the lozenge tilings to nonintersecting lattice 
paths, and subsequently applying the Lindstrom-Gessel-Viennot theorem (stated here 
as Lemma |T¥) . 



From lozenge tilings to nonintersecting lattice paths. There is a well-known trans- 
lation of lozenge tilings to families of nonintersecting lattice paths. We start with a 
lozenge tiling of the cored hexagon (see Figure |S[a). We mark the midpoints of the 
edges along the sides of length a and a + m and along the side of the triangle which is 
parallel to them (see Figure |8].b). Now, in the same way as in the proof of Theorem [7] 
in the preceding section, we connect these points by paths which "follow" along the 
lozenges of the tiling, as is illustrated in Figure ||b. Clearly, the resulting paths are 
nonintersecting, i.e., no two paths have a common vertex. If we slightly distort the 
underlying lattice, we get orthogonal paths with positive horizontal and negative ver- 
tical steps (see Figure ||.c). In the case that a, b and c have the same parity, we can 
introduce a coordinate system in a way so that the coordinates of the starting points 
Ai and end points Ej are 

Ai — (i — 1, c + m + i — 1), i — 1, 2, . . . , a, (5.1a) 

f a + b a + c \ 
Ai = I — h i — a — 1, — h i — a — 1 J , % = a + 1, a + 2, . . . , a + m, (5.1b) 

Ej = (b + j j = 1,2,... ,a + m, (5.1c) 
see Figure |[c. 

Suppose now that the parity of a is different from that of b and c, which is the case in 
Theorems |] and [|. Since in this case the core is slightly off the "truly central" position 
(because the triangle in the "truly central" position would not be a lattice triangle; see 
the definitions in the Introduction), the starting points of the lattice paths originating 
at boundary points of the core are changed slightly as well. The starting and ending 
points become 

A4 = (i — 1, c + m + i — 1), i = 1, 2, . . . , a, (5.2a) 

' a + b — 1 a + c— 1 
Ai = h % — a — 1, h i — a — 1 , i = a + 1, a + 2, . . . , a + m, 



(5.2b) 

Ej = {b + j - 1, j - 1), j = 1, 2, . . . , a + m. (5.2c) 

In either case, the lozenge tiling can be recovered from the path family, so that it 
suffices to count the families of nonintersecting lattice paths with the above-mentioned 
starting and end points. 

From nonintersecting lattice paths to a determinant. In order to count these families 



of nonintersecting lattice paths, we make use of a result due to Lindstrom |23], Lemma 1] 



and independently to Gessel and Viennot 0, Theorem 1]. In fact, it is the not so well- 



known general form of the result which we need here. In order to state this result, we 
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c + m 





a. A lozenge tiling of the cored hexagon in Figure |l] b. The corresponding path family 




c. The path family made orthogonal 



Figure 8. 



introduce some lattice path notation. We write V(A — > E) for the number of paths 
starting at A and ending at E. Given two sets A = {Ax, . . . , A n } and E = {Ex, . . . , E n } 
of lattice points and a permutation a, we write V(A — ► E CT) nonint.) for the number 
of families of n nonintersecting paths with the zth path running from A^ to E a n\, 



% — 1, 2, . . . , n. 



Now we can state the main result on nonintersecting lattice paths (see |23|, Lemma 1] 
or Theorem 1]). 

Lemma 14. Let Ax, A 2 , . . . , A n , Ex, E 2 , . . . , E n be points of the planar integer lattice. 
Then the following identity holds: 



det (V(A i ^E j ))=J2 



sgna) • V(A — > E CT , nonint.). 



(5.3) 
□ 



Remark. The result in [p3~| |, respectively |T^|, is in fact more general, as it is formulated 
for paths in an arbitrary oriented graph. But then the graph must satisfy an acyclicity 
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condition. We have not mentioned it in the formulation of the above lemma as it is 
automatically satisfied in our more restricted setting. 

Usually, this lemma is applied in the case that the only permutation for which non- 
intersecting lattice paths exist is the identity permutation, so that the sum on the 
right-hand side reduces to a single term, which counts all families (P\,P2, . . . ,P n ) of 
nonintersecting lattice paths, the ith path P$ running from to Ei, i = 1,2, ... ,n. 
(The only exceptions that we are aware of, i.e., applications of the above formula in 
the case where the sum on the right-hand side does not reduce to a single term, can 
be found in ||, pcjfl , and f4"Ifl .) This is, however, not exactly the situation that we 
encounter in our problem. Therefore, it seems that Lemma [L4| is not suited for our 
problem. However, our choice of starting and end points (see Figure §.c) implies that 
nonintersecting lattice paths are only possible if m consecutive end points (m being the 
side length of the equilateral triangle removed from the hexagon) are paired with the 
starting points from the triangle. So the corresponding permutation a, which describes 
in which order the starting points are connected to the end points, differs from the 
identity permutation by a composition of cycles of length m + 1. Thus, if m is even, 



we have sgna = 1, so that the right-hand side in Lemma |14] counts exactly all nonin- 
tersecting lattice path families and, thus, all the lozenge tilings that we are interested 
in. 

On the other hand, if m is odd, then the sign of the permutation a will not be 1 
always. In fact, as is straightforward to see, the sign of a is 1 if the number of paths 
which pass the core on the right is even, and is —1 otherwise. If this is translated 
back to the original lozenge tiling, T say, then it follows that sgn a is exactly equal to 
(— l) n ( T \ with the statistic n(.) from Section 0. Thus, in the case that m is odd, the 
determinant in Lemma |T^ gives exactly the (— l)-enumeration of our lozenge tilings. 

Since the number of paths from (x\, y{) to (x2, 2/2) with positive horizontal and neg- 
ative vertical steps equals the binomial coefficient \~^-x-i~) > our findings so far can 
be summarized as follows. 

Lemma 15. Let a, b, c, m be nonnegative integers, a, b, c having the same parity. Ifm is 
even, then the number of lozenge tilings of a hexagon with sides a, b+m, c, a+m, b, c+m, 
with an equilateral triangle of side length m removed from its center, equals 



( fb + c + m\ \ 

11 1 1< i < a » 



det 

l<i ,j<a+m 



b-i+j 

b+c 
2 



a+l<i<a+m 



(5.4) 



Ifm is odd, then the weighted count l) n ^ T \ where T varies through all the above 
lozenge tilings, is equal to the above determinant. 



Lemma 16. Let a, 6, c, m be nonnegative integers, a of parity different from the parity 
of b and c. If m is even, then the number of lozenge tilings of a hexagon with sides 
a,b + m, c, a + m,b,c + m, with an equilateral triangle of side length m removed that is 
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c + m 




FIGURE 9. A lozenge tiling of a hexagon with sides a = 5, 6 = 3, c = l and 
removed triangle of side length m = 2 and the corresponding paths. 

"central" in the sense that was described in the Introduction, equals 

\ 



det 

l<i,j<a+m 



I jb + c + m 
\b-i+j 

b+c 
2 

fe+q+1 



-l+J 



Ki<a 



O + l < i < a + m 



(5.5) 



If m is odd, then the weighted count 1) > where T varies through all the above 
lozenge tilings, is equal to the above determinant. 



6. POLYNOMIALITY OF THE NUMBER OF LOZENGE TILINGS 

The goal of this section is to establish polynomiality in m — the side of the core — of 
the weighted counts of lozenge tilings considered in Theorems [L], Q, £|, [5], provided a, b, c 
are fixed. Below we just address the case that a, b and c have the same parity (i.e., the 
case considered in Theorems |l] and the other case being completely analogous. 

We set up a bijection between the lozenge tilings of our cored hexagon and nonin- 
tersecting lattice paths in a manner different from the one in the preceding section. 
We start by extending all sides of the removed triangle to the left (if viewed from the 
interior of the triangle; see Figure |9], where these extensions are marked as thick seg- 
ments). These segments partition the cored hexagon into three regions. Furthermore, 
the segments cut some of the lozenges in two. (In Figure [] these lozenges are shaded.) 
In each of the three regions, we mark the midpoints of those edges of the "cut" lozenges 
and of those edges along the border of the region that are not parallel to the "thick" 
segments bordering this region (see Figure ||). Now, in each of the three regions, we 
connect the marked points by "following" along the lozenges of the tiling, in the same 
way as in Section |] (in the proof of Theorem |7|), and in Section |5| (see Figure §.b). The 
lozenge tiling can be recovered from the three nonintersecting path families. Thus this 
defines indeed a bijection. 
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Hence, if we fix the lozenges that are cut in two by the segments, the corresponding 
number of lozenge tilings which contain these fixed "cut" lozenges is easily computed 
by applying the Lindstrom-Gessel-Viennot theorem (Lemma [H]) to each of the three 
regions separately. This gives a product of three determinants, one for each region. The 
total number of lozenge tilings is then obtained as the sum over all possible choices of 
"cut" lozenges (along the segments) of this product of three determinants. 

It is easy to see that each entry in any of the three determinants is a binomial 
coefficient of the form ( m ^ x ), where x and y are independent of m. So the entries are 
polynomials in m, and, hence, the determinants as well. The segment which extends 
the side of the removed triangle that is parallel to a has length min{^p, which is 
independent of m, similarly for the other lines. The total number of lozenge tilings is 
thus equal to a sum of polynomials in m, where the range of summation is independent 
of m. Therefore it is itself a polynomial in m, as was claimed. 

Basically, the same arguments hold also for (— l)-enumeration. The only difference is 
that each product of three determinants is multiplied by a sign, depending (according to 
the definition of our statistic n) on the parity of the number of lozenge sides contained 
in the northeastern extension of the bottom side of the core. However, this number 
equals the length of this extension minus the number of lozenges the extension cuts 
through, and is therefore again independent of m. 



7. Determinant evaluations, I 



In this section we evaluate the determinant in Lemma [Tj^. The underlying matrix is 
a mixture of two matrices. If we would have to compute the determinant of just one 
of the matrices (i.e., if we consider the case a = or m = 0), then the determinant 
could be easily evaluated (see (|12.5| )). However, the mixture is much more difficult to 
evaluate. As it turns out, we have to distinguish between several cases, depending on 
the parities of a and m. 

It is convenient to take (b + c + m)\/ {b + a + m — i) \ (c + m + i — 1)! out of the ith 
row, i = 1,2, 
a + 1, a + 2, . 



and (£±£)!/(fc^2 + TO -i)! (£z2 + i-l)! out of the ith row, 



a + m. This gives 



/ fb + c + m 



det 

l<i ,j<a+m 



-i+j 

b+c 
2 



1 < i < a 



n 



i+j 
(6 4 



a+l<i<a+m 



C + m) 



\ (b + a + m-i)\(c + m + i- 1)! .J-* (^ + m 

1 v ' v ' i=a+l v 2. 



a+m 

n 



1)' 



x det 

l<i ,j<a-\-m 



(c + m + i-j + (b-i+j + l) a +m-j 



1 < i < a 
a < i < a + m 



(7.1) 



Thus it suffices to evaluate the determinant on the right-hand side. The advantage 
is that this determinant is a polynomial in b and c. This enables us to apply the 
"identification of factors" method, as proposed in [19|, Sec. 2.4]. The four lemmas 
below address the four different cases, as a and m vary through all possible parities. 



LOZENGE TILINGS OF HEXAGONS WITH A CENTRAL TRIANGULAR HOLE 



23 



Lemma 17. Let a and m be both even nonnegative integers. Then 



= IffSSff n (i + *>u (t + *)U if (»+c +m+ 2k + d«— ■ 

V 2 ' fc=l fc=0 

a/2-1 m m/2 

x j [ (6 + c + 2m + 2k) a ~ 2k j [ (6 + c + 2A;) a+m - fc JJ(6 + c + 2A;) m - fc . (7.2) 

&=1 fc=m/2+l fc=l 

Proof. Let us denote the determinant in ( |7.2|) by -Di(6, c). 

We proceed in several steps. An outline is as follows. The determinant Di (b, c) is 
obviously a polynomial in b and c. In Steps 1-5 we show that the right-hand side of 
( |7.2[) divides c) as a polynomial in b and c. In Step 6 we show that the degree of 
D 1 (b, c) as a polynomial in b is at most ( a+ ™) . Of course, the same is true for the degree 
in c. On the other hand, the degree of the right-hand side of ( [7.2| ) as a polynomial in b 
is exactly ( a ^ m ). It follows that Di(b, c) must equal the right-hand side of ( |7.2|) times a 
quantity which does not depend on b. This quantity must be polynomial in c. But, in 
fact, it cannot depend on c as well, because, as we just observed, the degree in c of the 
right-hand side of ( |7.2|) is already equal to the maximal degree in c of D\ (b, c) . Thus, 
this quantity is a constant with respect to b and c. That this constant is equal to 1 is 
finally shown in Step 7, by evaluating the determinant Di(b, c) for b = c = 0. 

Before we begin with the detailed description of the individual steps, we should ex- 
plain the odd looking occurrences of "e = a mod 2" below (e.g., in Step l(a)-(d)). 
Clearly, in the present context this means "e = mod 2", as a is even by assump- 
tion. However, Steps 1-6 will also serve as a model for the proofs of the subsequent 



c 



+ m + i-j + 1)^1 (b-i+j + l) a +m-j 1 < i < 



Lemmas |lq-|20l. Consequently, formulations are chosen so that they remain valid with- 
out change at the corresponding places. In particular, in the context of the proofs of 
Lemmas [T8| and EOl the statement "e = a mod 2" will mean "e = 1 mod 2" . 



Step 1. YYhii (I + ^)a/2 (f ~^ ^)a/2 divides the determinant. The original determinant 
is symmetric in b and c for combinatorial reasons. The factors which were taken out of 
the determinant in ( |7.1|) are also symmetric in b and c (this can be seen by reversing all 
the products involving c) . Therefore it suffices to check that the linear factors involving 
b divide Di(b,c), i.e., that the product Y]!k=i (I + ^) Q /2 divides Di{b,c). 

We distinguish between four subcases, labeled below as (a), (b), (c), and (d). 

(a) (b + e) e divides Di(b,c) for 1 < e < min{a, m}, e = a mod 2: This follows from 
the easily verified fact that (b + e) is a factor of each entry in the first e columns of 
D 1 (b,c). 

(b) (b + e) m divides D^b, c) for m < e < a, e = a mod 2: We prove this by finding 
m "different" linear combinations of the columns of Di(b,c) which vanish for b = —e. 
By the term "different" we mean that these linear combinations are themselves linearly 
independent. (Equivalently, we find m linearly independent vectors in the kernel of the 
linear operator defined by the matrix underlying Di(—e, c).) See Section 2 of fl8||, and 



in particular the Lemma in that section, for a formal justification of this procedure. 
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To be precise, we claim that the following equation holds for s = 1,2, ... ,m, 

e+s—m 



E 



e — m + s — l\(c + a — e — s + 2m + l) e+ 
J-l 



(a — e — s + 2m + 1 



' "' (column j of Di(—e,c)) = 0. 

(7.3) 



e+s—j—m 



Since the entries of Di(b,c) have a split definition (see (|7.2|) ), for the proof of the 
above equation we have to distinguish between two cases. If we restrict ( |7.3| ) to the ith 
row, i < a, then ( |7.3| ) becomes 



E 

3=1 



e — m + s — l\(c + a — e — s + 2m + l) e + s -j-m 
J-l 



(a - e - s + 2m + l) e+s _,-_ m 
■{c + m + i-j + (~e-i + j + l) a+m -j = 0, (7.4) 
whereas on restriction to the ith row, i > a, equation (|7.3|) becomes 

e+s—m 



E 



e - m + s - l\(c + a - e - s + 2m + l) e+a _ J _„, 
J-l 



(a — e — s + 2m + 1) 



e+s—j—m 



0. (7.5) 



First, let i < a. Here and in the following, we make use of the usual hyper geometric 
notation 



, . . . , u, r 



«1 



6i, ... ,6, 



; 2 



E 

fc=0 



(ai)^ ■ • ■ {a r )k k 
k\ {b l )k---{b s )k 



(7.6) 



In this notation, the sum on the left-hand side of (17.41) reads 



(2-e-i) 



-l+a+m 



'1 + a + c — e + 2m — s) 



-l+e—m+s 



[l + a-e + 2m - s)_ 1+e _ m+s 

1 — c — i — m, 1 — e + m — s, 1 — a — m 



x 3^2 



1— a — c — m, 2 — e — i 



Next we use a transformation formula due to Thomae (see also |I| (3.1.1)]), 



3-^2 



A,B,-n 
D, E ' 



"3^2 



— n, B,D — A 
DA + B — E — n' 



(7.7) 



where n is a nonnegative integer. This gives 

(l + a + c- e + 2m - s) e _ m+s _i (1 - i - m + s) a+m _i 



(1 + a - e + 2m - s) e _ m+s _! 

X 3^2 



1 — a — m, 1 — e + m — s, —a + i 
1 — a — c — m, 1 — a + i — s ' 



The factor (1 — i — m + s) a+m _i vanishes for i < a and the denominator is never zero, 
so the sum in (|7.4j ) equals zero, as desired. 
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We proceed similarly in order to prove ( |7.5| ) for i > a. The hypergeometric form of 
the sum in ( |7.5| ) is 



a+m- 



_i(l + a + c- e + 2m- s) e _ m+s _i 



1 + a - e + 2m- s) e _ m+s _i 



1 + - - - 

- 1 ' 2 2 



i,l — a — m,l — e + in — s 



1 — a - c-m, 2 + | — | 



;1 



Using the transformation formula ( |7.7| ) again, we get 



l + ^ — | — 2 + ?77,) e _ m + s -l 



l + f + f- m-i + S) a+2 m-e 

(l + a + c- e + 2m- s) e _ m+s _i 



'l + a- e + 2m-sl 



-l+e— m+s 



X 3^2 



1 — e + m — s, 1 — a — m, — 4f — § + z — m 
l — a — c — m, 1 — ^-§+i — s 



This expression is zero, because the factor (1 + |+|— m— z+s) a+2m - e -s vanishes for z > a 
(it is here where we need e = a mod 2, because this guarantees that 1 + 1 + 1 — m — z + s 
is an integer). So the sum in ( |7.5| ) equals zero, as desired. 

(c) (b + e) a divides Di(b,c) for a < e < m, e = a mod 2: Proceeding in the spirit 
of case (b), we prove this by finding a linear combinations of the columns of Di(b,c) 
which vanish for b = — e. To be precise, we claim that the following equation holds for 
, a: 



S = l,2, 



3=1 



1\ (f + 



+ 1 )(e- a 



(e-a)/2+s-j 



(m + 



3a— e 



+ 1) 



(e-a)/2+s-j 



■ (column j of D\{— e, c)) = 0. 

(7.8) 



In order to prove this equation, we first restrict it to the zth row, % < a. Then, in 
hypergeometric notation, the left-hand side reads 



(2-e-i) 



i-i (1 + a + § — % + m — s)_i° + « 



[1 + ^-l+m-s) 



-!-!+§- 



X 3^2 



1 — a — m, 1 — c — i — m, 1 + ^ — 



I - § - § - m, 2 - e - z 2 ' 1 



We apply the transformation formula ( |7.7|) and get 



C 6 

. 1 (l + o+--- + m-s) 



Z + S) 3a 



2- 2+m-s 

3-^2 



+ 771- Sj_!_a +# . 



^ 2 2 X C Z TYLj 2 2 1 

l-f-f-m,l + f- c+ f- m-s ; 



This expression is zero because the factor (1 — | — | — z + s)3a. 



vanishes. 
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If instead we restrict the left-hand side of (|7.8|) to the ith row, i > a, and convert it 
into hypergeometric form, then we obtain 

(2 + f - § - i) a+m -i (1 + a+ f- f + m- s)_i_§+| +s 



: i + ¥-f+^- s )-i-f + i- 



We apply again the transformation formula (|7.7|) . This gives 
(1 + a + f- f + m- + s)-i +a+Tn 



- f - i,l - a - m, 1 + f - f - s 
l-|-f-m,2 + f-f-i ;i 



l + f -f + m- S )_ 1 _ f+f+s 



x 3^2 



1 + f — f — s, 1 — a — m, —a + i — m 
1 — f — f — m, 1 — a + i — m — s ' 



This expression is zero because the factor (1 — i + s)_i +a+m vanishes for a + 1 < z < 
a + m. So the sum in ( [7.8|) equals zero, as desired. 

(d) (b + e ) a + m ~ e divides -D^o, c) for max{a, m} < e < a + m — 1, e = a mod 2: Still 
proceeding in the spirit of case (b), this time we find a + m — e linear combinations of 
the rows of .Di(6, c) which vanish for b = — e. To be precise, we claim that the following 
equation holds for s = 1,2, . . . , a + m — e: 



E 



S _A (-^(fizs + l) (^ + m) j _ 1 



(1 + c - e + m) a+m _ s+i _i 

• (row (a + m — e — s + z) of Di(— e, c)) 

+ (row (m + f - f - s + 1) of Di(-e, c)) = 0. (7.9) 

In the sum, it is only the first a rows which are involved, whereas the extra term is 
a row out of the last m rows of the determinant. Therefore, by restriction to the jth 
column, we see that it is equivalent to 



E 



i=i 



S _A (-l)*(^ + l) a+m _ s f^+m)^ 



i — lj (l + c-e + m) a+m _ s+ i_i 

•(a + c + 2m-e-s + i-j + (-a - m + s- i + j + l) a+m _j 

+ (2fS + a + m- s-j + 2) (-a - m + s + j)a+ m -j = 0. (7.10) 

We treat the cases j < a + m — s and j > a + m — s separately. For j < a + m — s 
the factor (—a — m + s — i+j + l) a+m _j, which appears in the sum, is zero for all 
the summands, as well is the factor (—a — m + s + j) a+m _p which appears in the extra 
term in ( |7.10| ). 



For j > a + m — s we convert the sum in ( |7.10| ) into hypergeometric form and get 



-(1 + f - §)a+m-s (2 + a + c- e-j + 2m - s)_i +i (-a + j - m + s) a _ iH 



l + c-e + m) a+m _ 
x 2^1 



f-f + m, 1 + a—j + m — s 
2 + a + c — e — j+2m- s ' 
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We can evaluate the 2-^i-series by the Chu-Vandermonde summation formula (see 
[35], (1.7.7), Appendix (III.4)]), 

(C-A) n 



A, -n 

C ' 



(C) ? 



(7.11) 
(7.12) 



where n is a nonnegative integer. Thus we get 

-(2 + a + § - § - j +m- s)j-_i {-a+j -m + s) a _ j+m . 

It is easily seen that adding the extra term in (|7.10| ) gives zero. 

Step 2. YII^q 1 (b + c + m + 2k + i) a - 2fc - 1 divides the determinant. We find e + 1 
linear combinations of the rows of Di(b, c) which vanish for b = — c — a — m + l + e. To be 
precise, we claim that the following equation holds for < e < a — 2, s = l,2,...,e + l: 

a— e— 1 



£ 

i=l 



(C + m + 2) a _ e _ i+s _i 



a — e — 2 

1 + i) a _ e _i +s _i V i-1 7 (s - z + a - e - l)(a 
(row i of -Di(— c — a — m + 1 + e, c)) 



Sja-e-l 



2)! 



+ (-1 



ia- e— 1 



^row (a-e-l+s) of £>i(-c - a - m + 1 + e, c)) = 0. (7.13) 



Restricted to the jth column, and converted into hypergeometric notation, the sum 
in ( |7.13| ) reads 



1 — a — c + e + j 



m 



a—j+m 



(1 + c + m) a . 



e+s-2 



[2 + C~j + m)j-i {s) a -e-2 



(1 

X 3^2 



a-e-2 \ c — e Ja-e+s-2 

2 — a + e — s, a + c — e — j + m, 2 — a + e 
3 — a + e — s,2 + c — j + m 



;1 



Here we use the Pfaff-Saalschiitz summation formula (see |^5|, (2.3.1.3), Appen- 
dix (III.2)]) 

(C - A) n (C - B) n 



3-^2 



A, B, -n 
CA+A + B-C-n : 



(C) n (C-A- B) 7 



(7.14) 



where n is a nonnegative integer. Thus we get 



\a— e— 1 , 



1 + C + m)_ 2 +a-e+s (2 + c - j + m)_ 1+j 



X 



(3-2a-c + 2e + j- m- s)„ 2 +a- 



C - e)_ 2+ a_ e+s (1 - C + e)_ 2 -e+i-m 

It is easily verified that adding the jth coordinate of the extra term in (|7.13j ) gives zero, 
as desired. For now, we need equation (|7.13|) only for even e. 

Step 3. rifc^i" 1 (b + c + 2m + 2k) a ~ 2k divides the determinant. We find e linear com- 
binations of the columns of Di(b,c) which vanish for b = — c — 2m — a + e. To be 
precise, we claim that the following equation holds for 0<e<a, e = a mod 2, and 
e: 

a+m+s—e / \ 

Ei q rn e \ 
( . j • (column j of D\{—c — 2m — a + e, cj) = 0. (7-15) 

j=s 



8 = 1,2, 



3-8 



2S 
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Restricted to the ith row, i < a, and converted into hypergeometric notation, the 
left-hand side sum in ( |7.15| ) reads 

(1 + c + i + m — s) s _i (1 — a — c + e — i — 2m + s) a+m _ s 

„ — c — i — m + s, —a + e — m ., 

x 2 -Pi 1 • ; 1 

1 — a — c + e — z — zm + s 

This is summable by the Chu-Vandermonde summation formula ( 7.11|) . We get 

(1 - a + e - m) a _ e+m (1 + c + i + m - s) s _i 
(1 — c + e — z — m) s _ e 

This expression equals zero because the factor (1 — a + e — m) a _ e+m 



vanishes. 



On the other hand, if i > a, the left-hand side sum in ( |7.15| ), restricted to the z'th 
row and converted into hypergeometric from, reads 



2 2 ~~ ^ ~ TTl -\- Sja+m- 



X 2^1 



— | — z + s, —a + e — m 
f — i — m + s ' 



1 - 



The Chu-Vandermonde summation formula (|7.11|) turns this expression into 



,1 ~ f + f — ™)a-e+m (1 # + « S y 



s-1 



(l + a-f + |- z)-e+s 

This expression is zero because the factor (1 — | + | — m) a _ e+m vanishes for e = a mod 
2. So the sum in (|7.15|) is zero, as desired. 

Step 4- rifclm/2+1 ip + c + 2k) a+m ~ k divides the determinant. We find a + m — e linear 
combinations of the columns of Di(b, c) which vanish for b = — c — 2e. To be precise, we 
claim that the following equation holds for m/2 < e < m and s = 1,2,. . . ,a + m — e: 



I ) • (column j of c — 2e, c)) = 0. 

3=a ^ 



(7.16) 



Restricted to the ith row, i < a, and converted into hypergeometric notation, the 
left-hand side sum in (|7.16|) reads 



'1 + c + z + m — s) s _i (1 — c — 2e — z + 



a+m—s 2 



Ft 



— c — i — m + s, —e_ 
l-c-2e-i + s' 



The result after application of the Chu-Vandermonde summation formula ( |7.11 

(1 — 2e + m) e (1 + c + i + m — s) s _i 



is 



'1 + a — c — 2e — z + m) 



-a+e—m+s 



This expression equals zero because the factor (1 — 2e + m) e vanishes. 

On the other hand, if i > a, the left-hand side sum in (|7.16|) , restricted to the ith 
row and converted into hypergeometric from, reads 



-i(i + ! 



i + s) 



a+m—s 2 



l + f-f-e-i + s' 
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Chu-Vandermonde summation ( |7. 1 1| ) yields 

a 

2 1 2 



l-e) e (l-§ + | + z 



1 + ^ — | — e — z + m)_ a+e _ m+s 



This expression is zero because the factor (1 — e) e vanishes. So the sum in ( |7.16| ) is 
zero, as desired. 

Step 5. rifc=i + c + 2k) m ~ k divides the determinant. We find e linear combinations 
of the rows of D\(b, c) which vanish for b = — c — 2m + 2e. To be precise, we claim that 
the following equation holds for e < m — 1 and s = 1, 2, . . . , e: 



m-s+l / \ ( c , a I -\ 



(f-f-e + z) 

■ (row (a + i) of Di(-c - 2m + 2e, c)) = 0. (7.17) 

Restricted to the jth row, and converted into hypergeometric notation, the left-hand 
side sum in ( |7.17| ) reads 



[l + § + i) m -s (2 + § + § - jVi (-f - f + e + 3 



m 



a—j+m 



(l-| + l-e) 



x 2 Fi 



1 + f + f- e- j + m, -m + s_ 



2+|+i-j 

After applying Chu-Vandermonde summation ( [7.11]) again, we obtain 
- (1 + f + f ) m _ s (l + e - m) m _ s 

x ~ I + e + J ~ ^)a-j+m(2 + f + I - j + m- .s)i- » 



V- 1 - 2 2 ^jm—s 

This expression equals zero because the factor (1 + e — m) m _ s vanishes. So the sum in 
( |7.17Q is zero, as desired. 

Step 6. Determination of the degree of Di(b,c) as a polynomial in b. Obviously the 
degree of the (i, j)-entry of Di(b, polynomial in b is a + m — j. Therefore, if we 

expand the determinant D\ (b, c) according to its definition as a sum over permutations, 
each term in this expansion has degree ( a t, m ) in b. Hence, D\(b, c) itself has degree at 
most ( a+ 2 rn ) in b. 

Step 7. Computation of the multiplicative constant. As we observed at the beginning 
of this proof, Steps 1-6 show that the determinant D\ (6, c) is equal to the right-hand 
side of (7.2p up to multiplication by a constant. To determine this constant, it suffices 



to compute Di(b, c) for some particular values of b and c. We choose b = c = 0. The 
value of -Di(0, 0) is most easily determined by going back, via ( [7.1[ ) and Lemma |T5], to 
the origin of the determinant D\ (b, c) , which is enumeration of lozenge tilings. Figure pH| 
shows the typical situation for b = c = 0. As the figure illustrates, there is exactly one 



lozenge tiling of the region. Hence, by Lemma |15| it follows that the determinant (|5.4|) 
must be equal to 1 for b = c = 0. If we substitute this into (|7.1|) , we have evaluated 
Di(b,c), which is the determinant on the right-hand side of (|7.1|) , for b = c = 0. It is 
then a routine task to check that the result agrees exactly with the right-hand side of 
T2l) for b = c = 0. 
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m < 




FIGURE 10. The unique lozenge tiling for b = c = 



This completes the proof of the theorem. 



□ 



Lemma 18. Let a and m be nonnegative integers, a odd and m even. Then 



Hc + m + i-j + (b - i + j + l) a + m -j 



' a+m—j 



l<i<a 
a < i < a + 1 



H(a + m)H( 



o-l - 
2 - 



H (^±l)H(f) 2 



pj-^ a+m-l -j ^-^ a+m+l 'j 2m(a+m-l)/2 
m/2 

n(¥ + *) WI)/2 (¥ + *),. 



X 11 + fc J(a+l)/2 IT + fc J(a-l)/2 (V" + k ) (a+l)/2 + fc ) 

fc=l 

(o-l)/2-l (o~l)/2 

x | [ (b + c + m + 2k + l) a - 2k - 1 Yl (b + c + 2m 



(a-l)/2 



2fc) 



fc=0 



fc=l 



m/2 



x ]"[ (b + c + 2A;) a+m ~ fc JJ (6 + c + 2fc) 

fc=m/2+l fc=l 



m—k 



(7.18) 



Proof. We proceed analogously to the proof of Lemma The only difference is the 
parity of a, so we have to read through the proof of Lemma [17| and find the places 
where we used the fact that a is even. 

As it turns out, the arguments in Steps 1-5 in the proof of Lemma |17| can be used 
here, practically without change, to establish that the right-hand side of ([7.18 ) divides 
the determinant on the left-hand side of (|7.18|) as a polynomial in b and c. Differ- 
ences arise only in the products corresponding to each subcase (for example, the ar- 
guments in Step 3 of the proof of Lemma pj] prove that rife=i 1 + c + 2m + 2k) a - 2k 
divides the determinant D\ (b, c) if a is even, while for odd a they prove that 
ni'!^ 1 ' 1 '' 2 (b + c + 2m + 2k) a ~ 2k divides Di(b, c)), and in the fact that in Step 2 we are 
now interested in the factors corresponding to odd values ofe, 1 < e < a — 2 (because 
here the factors with even e are covered by Steps 3 and 4). 
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FIGURE 11. A lozenge tiling and the corresponding path family for b = c=l,a = 5 



Also Step 6, the determination of a degree bound on the determinant, can be used 
verbatim. 

For the determination of the multiplicative constant relating the right-hand and the 
left-hand side of ( [7.18| ), we have to modify however the arguments in Step 7 of the proof 
of Lemma [17]. We determine the constant by computing the determinant for b = c = 1. 
Again, this value is most conveniently found by going back, via ( |7. 1| ) and Lemma [15 



to the combinatorial root of the determinant, which is enumeration of lozenge tilings. 
We claim that the number of lozenge tilings for b = c = 1, a odd and m even, equals 



2 ^m + l + 2fi 



q-l 
2 



(7.19) 



This can be read off Figure [IT], which shows a typical example of the case b = c = 1: 
The path starting at Aa+i (see the labeling in Figure [11]; it is derived from the labeling 
of starting points of paths in Figure |8]) must pass either to the right or to the left of 
the triangle. Since the hexagon is symmetric, we can count those path families where 
the path passes to the right, and in the end multiply the resulting number by two. For 
those path families, the paths starting at points to the right of Aa+i are fixed. The 

2 

paths to the left have all exactly one South-East step. Suppose that the South-East 
step of the path which starts in Ai, 1 < i < (a — l)/2, occurs as the /ijth step. Then 
we must have 

m + 2>h 1 >h 2 >---> ha^i > 1. 

2 

So we just have to count monotonously decreasing sequences of 2== numbers between 
1 and m + 2. The number is exactly the binomial coefficient in (|7.19| ). It is then a 
routine task to check that, on substitution in (|7. 1|) , the result agrees exactly with the 
right-hand side of ( |7.18|) for b = c = 1. □ 
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Lemma 19. Let a and m be nonnegative integers, a even and m odd. Then 

f(c+ m + i - j + l)j_i (6 - % + j + l) a +m-j l<i<a 
l<mi + m {(^+i-3 + + l) Q+m _, a<i<a + m 

_ ( /2 H(a + m)H(f) 2 H(^)H(^) 

s. / tt t g+m— 1 *\ TX t a+rra+1 ' 



jj^ g+ro-1 ^ ^m{a+m-\) 1 2 



2 ^ "V 2 

(m-l)/2 (m-l)/2 a/2-1 

x(i+^) f (f+^) f n (!+ fc ) 2 f n d+A:) 2 f n ( & + c+2fc +-) a " 2fc 

fc=l fc=l k=l 

a/2-1 (m-l)/2 m 

JJ (6 + c + 2A; + 2m) a - 2fe J] (1 + 6 + c + 2 jfe + m) a JJ (6 + c + 2 A;) m " fc . 



x 

k=l k=0 k=l 



(7.20) 

Proof. We proceed analogously to the proof of Lemma 0. The only difference is the 
parity of m, so we have to check the places in the proof of Lemma [T7| where we used 
the fact that m is even. 

Again, Steps 1-6 can be reused verbatim, except that the products corresponding to 
the individual subcases are slightly different, and in Step 2 we are now interested in the 
factors corresponding to odd values ofe, 1 < e < a — 2 (because the factors with even 
e are covered by Steps 3 and 4). 

The computation of the multiplicative constant relating the right-hand and the left- 
hand side of (|7.2Qj) is done analogously to Step 7 in the proof of Lemma [T7|. I.e., we 
compute the determinant for b = c = by going back, via ( |7.1[ ) and Lemma |I^, to 
the lozenge tiling interpretation of the determinant. We already concluded in the proof 
of Lemma |T7] that for b = c = there is just one lozenge tiling (see Figure [TIT). By 
definition, the statistic n(.) attains the value a/2 on this lozenge tiling, so that its 
weight is (— l) fl//2 . It is then not difficult to verify that, on substitution of this in ( |7.1| ), 
the result agrees exactly with the right-hand side of ( [7.20[ ) for b = c = 0. □ 

Lemma 20. Let a and m be odd nonnegative integers. Then 

f {c + m + i- j + l) i _i(6-i+ j + l) a+m _ j i<i<a \_ 

l<S+ m +i ~ j + 1) (*±* - i +j + l) a+m ^ a < i < a + m ) - U ' 

Proof. Analogously to the previous cases, we can show that the product 

(m+l)/2 (m-l)/2 0-l)/2 

n (¥+*)(a + i) /2 n n (b+c+ m +2kr- 2k 

i=l i=l k=l 

(o-l)/2 m (m-l)/2 

x I [ (b + c + 2m + 2kf- 2k \\ {b + c + 2k) a+m - k \ \ {b + c + 2k) m ~ k 

k=l fc=(m+l)/2 k=l 

divides the determinant as a polynomial in b and c. Although not completely obvious, 
this is implied by the linear combinations of Lemma [T7|, Steps 1-5. The degree in 
b of this product is ( a ^ m ) + 1 which is larger than the maximal degree ( a ^ m ) of the 
determinant viewed as a polynomial in b. So the determinant must be zero. □ 
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8. Determinant Evaluations, II 

In this section we evaluate the determinant in Lemma [16]. We proceed analogously 
to Section |7] and start by taking (b + c + m)\/(b + a + m — i) \ (c + m + i — 1)! out of 
the zth row, i = 1, 2, . . . , a, and (*±£) ! / (ttM±l + m -i)\ (<==|=i + i - l) ! out of the z'th 
row, i — a + 1, a + 2, . . . , a + m. This gives 



/ 



det 

l<i ,j<a+m 



i + c + m 
b-i+j 

b+c 
2 

b+a+1 



n 



{b + c + m)\ 



l<i< a 



a+l<i<a+m 



a+m 

n 



\ (b + a + m-i)\ (c + m + i - 1)! .j-^ (&±^±1 + m _i)l (£=|=i + j _ i)l 



x det 

l<i ,j<a+m 



(c + m + i-j + !)_,-_! (6 - z + j + !)«,+ 



( £= r i +^"-i+i) 7 - 1 ( 



b+a+l 



a+m— j 



l<i<a 
a < i < a + m 



5.r 



Thus it suffices to evaluate the determinant on the right-hand side. As in the pre- 
ceding section, the advantage is that this determinant is a polynomial in b and c. So 
we can again apply the "identification of factors" method, as proposed in |Tj| Sec. 2.4]. 
We note that the first a rows of the matrix are identical to those of ( |7.1| ), whereas the 
other m rows differ only slightly. Hence we can use many arguments from Section |7|. 
The four lemmas below address the four different cases, as a and m vary through all 
combinations of parities. 



det 



Lemma 21. Let a and m be both even nonnegative integers. Then 
(c + m + i- j + j + l)o+ 

(£=|=1 +i-j + 1) ,_ x (2±|±1 -i + j + l) 

k) a /2 



J + ( 

m/2 



a+m—j 



l<i<a 
a < i < a + m 



H(a + m)H(f) 2 H(f) 2 



/ a+m ^ 2 2"i(a+m-l)/2 
m/2 

x n (¥ + 

fc=i 

a/2-1 

x | [ (b + c + 2m + 2k) 

k=l 



, c+1 
/2 I — 



fc)a/2 



a/2-1 

+ k) a/2 ] [ (6 + c + m + 2fc + 1) 



a-2fc-l 



fc=0 



m/2 



a-2fc 



\a+»n— fc 



| | (b + c + 2k) 

fc=m/2+l k=l 



f[ (b + c + 2k) m ~ k . (8.2) 



Proof of Lemma \21. Let us denote the determinant in (|8.2| ) by D2(b,c). We will again 
proceed in the spirit of the proof of Lemma 0. I.e., we first show, in Steps 1-5 below, 
that the right-hand side of (|8.2j ) divides D2(b,c) as a polynomial in b and c. Then, in 
Step 6, we show that the degree of D 2 (b, c) as a polynomial in b is at most ( a+ 2 m ), the 
same being true for the degree in c. Analogously to the proof of Lemma [17], we conclude 
that D 2 (b, c) must equal the right-hand side of (|8.2|) , times a constant with respect to 
b and c. That this constant is equal to 1 is finally shown in Step 7, by evaluating the 
determinant D 2 (b, c) for b = c = 1. 
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In order to prove (in Steps 1-5) that the right-hand side of ( |8.2| ) divides D 2 (b,c), 
for each linear factor of ( |8.2| ) we exhibit again sufficiently many linear combinations of 
columns or rows which vanish. These linear combinations are almost identical (some- 
times they are even identical) with the corresponding linear combinations in the proof 
of Lemma |T7[ Consequently, we will merely state these linear combinations here, but 
will not bother to supply their verifications, because these parallel the verifications in 
the proof of Lemma |17[ 

Step 1. nri? (*r + fc W2 + k )a/2 (^ + fc) /2 (^ + k) a/2 divides the determi- 
nant. Unlike in the case of the previous determinant Di(b, c) (see ( |7.2|) ), here it is not 
possible to infer symmetry of D2(b, c) in b and c directly from the definition. Therefore 
it will be necessary to prove separately that the factors involving b, respectively c, divide 
the determinant. 

Again, we distinguish between four subcases, labeled below as (a), (b), (c), and (d). 

(a) (b + e) e (c + e) e divides D 2 (b,c) for 1 < e < min{a,m}, e ^ a mod 2: This follows 
from the easily verified fact that (b + e) is a factor of each entry in the first e columns 
of D 2 (b, c), respectively, that (c + e) is a factor of each entry in the last e columns of 
D 2 (b,c). 

(b) (b + e) m (c + e) m divides D 2 (b,c) for m < e < a, e ^ a mod 2: The following 
equations hold for s — 1, 2, . . . , m: 

E" ,_m fe-m + s- 1\ (c + a-e-s + 2m + l) e+ _ m . n 

{ j-l ) (a-e- S + 2m+lW,_ ro ■ (column j of C 2 (-e,c)) = 0, 

(8.3) 

and 

1\ (b + a-e-s + 2m + l) e+s -j-m 
J (a - e - s + 2m + l) e+8 _j_ m 

• (column (a + m + 1 - j) of D 2 (b, -e)) = 0. (8.4) 



e+s—m 

£ 

j'=i 



m + s 
J-l 



(c) (6 + e) a divides D 2 (b,c) for a < e < m, e ^ a mod 2: The following equations 
hold for s = 1, 2, . . . , a: 



e-a-l Lg 



2 



(e-o-l)/2+«-j 



■ , ^ j- 1 J (m + ^-s + i), ..... . 

j=l x 7 V 2 / (e— a— l)/2+s— j 

• (column j of D 2 (—e, c)) = 0, (8.5) 



and 



(e-o-l)/2+*-j 



^+ V j — l / (m + ^-s + 1), ..... . 

j=l x 7 V 2 / (e— a— l)/2+s— j 

• (column (a + m + 1 - j) of D 2 (6, -e)) = 0. (8.6) 
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(d) (b + e ) a + m - e divides D2(b,c) for max{a, m} < e < a + m, e ^ a mod 2: The 
following equations hold for s = 1,2, . . . ,a + m — e: 



s 



i-lj (l + c- e + m) a+m _ s+i _i 

• (row (a + m — e — s + i) of D 2 (—e, c)) 

+ (row + of D 2 (-e, c)) = 0, (8.7) 



»=i 

and 

s 

(1 + 6 - e + m) a+m „ s+i _i 

• (row (e + s — m — i + 1) of D 2 {b, — e)) 

+ (row (2±I + § + s ) of D 2 (b,-e)) = 0. (8.8) 

SYep n^Q -1 (6 + c + m + 2/c + l) ^ 2 ^" 1 divides the determinant. The following 
equation holds for < e < a — 2, s = 1, 2, . . . , e + 1: 

(c + m + i) a _ e _ i+s _i /a-e-2\ (s)a-e-l (-1) 1 



E 

i=i 



(c — e — 1 + i) a _ e _j +s _i \ i — 1 / (s — i + a — e — l)(a — e — 2)! 

■ (row i of D 2 (—c — a — m + 1 + e, c)) 
+ (-l)^ 6 " 1 ■ (row (a - e - 1 + s) of £> 2 (-c - a - m + 1 + e, c)) = 0. (8.9) 



Here, we need equation ( |3.9|) only for even e. 

Step 3. Ylt=i 1 (b + c + 2m + 2k) a ~ 2k divides the determinant. The following equa- 
tion holds for < e < a, e = a mod 2, and s = 1, 2, . . . , e: 

a+m+s—e ✓ \ 

j-v / a + m_e j . ( columnj of jD2 (_ c _ 2m _ a + e ;C )) = o. (8.10) 

j=8 V J s / 

SYep ^. rifcLm/2+1 (b + c + 2k) a+m ~ k divides the determinant. The following equation 
holds for m/2 < e < m and s — 1, 2, . . . , a + m — e: 

s+e / s 

( . 6 J • (column j of ,D 2 (-c - 2e, c)) = 0. (8.11) 

Step 5. nl^i {b + c + 2k) m ~ k divides the determinant. The following equation holds 
for e < m — 1 and s = 1, 2, . . . , e: 



m— s+1 



V ( - 1), (T-7) 



A (f + l + i-i) 



m— s— i+1 



,._J ^ " 1 / (2 2 e + 2 Dm-s-i+l 

■ (row (a + i) of D x (-c - 2m + 2e, c)) = 0. (8.12) 

Step 6. Determination of the degree of D 2 (b,c) as a polynomial in b. This is clearly 
the same degree as for Di(b,c), that is, ( a+ 2 m )- 
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Step 7. Computation of the multiplicative constant. In analogy to the proof of 
Lemma [Tj^ we evaluate the determinant for b = c = 1. Again, we do this by going 
back, via ( |S.1| ) and Lemma [H], to the combinatorial origin of the determinant, which is 
enumeration of lozenge tilings. We can still use Figure |TT| for our considerations. The 
number of lozenge tilings is easily seen to be equal to ( m+ l + ^) + ( m+ <*^?~ )■ ft i s then 
a routine computation to verify that this does indeed give the multiplicative constant 
as claimed in (18.2). □ 



Lemma 22. Let a and m be nonnegative integers, a odd and m even. Then 



( (c + m + i — j + (b — i+j + l) a +m-j l<i<a 
l<iS<*+m {{^r 1 !),_! ~i + 3 + l) a+m _, a< t <a + m 

H(a + m)H(^)H(^)H(f) 2 

pj^ q+m— 1 ^ j^^ a+m+l -j 2m(a+m-l)/2 
m/2 

II (I + k )^-l)/2 (| + *0(a+l)/2 (f + fc)(a-l)/2 (f + ^)(a+l)/2 



[ [ {b + c + m + 2k + i)*- 2 *- 1 j ] (fo + c + 2m + 2A;) a - 



2/v 



2 / "V 2 
m/2 

X 

fe=l 

(a-3)/2 (a-l)/2 

X 

fc=0 k=0 

m m/2 

x Yl (b + c + 2k) a+rn - k Y[(b + c + 2k) m - k . (8.13) 

fe=m/2+l fc=l 

Proof. We proceed analogously to the proof of Lemma ^l|. The only difference is the 
parity of a, so we have to check the places in the proof of Lemma |2l] where we used the 
fact that a is even. 

Steps 1, 3-5 can be reused verbatim, but the corresponding products are slightly 
different. 

In Step 2 we are now interested in the factors corresponding to odd values of e 
(1 < e < a — 2), because the factors with even e are covered by Steps 3 and 4. 
Step 6 can be reused verbatim. 

The computation of the multiplicative constant is done analogously to Step 7 in the 



proof of Lemma [L7|. Again using Figure [10], we see that the number of lozenge tilings, 
related to our determinant via ( |8.1| ) and Lemma [16|, for b = c = equals 1. It is then a 
routine computation to verify that this gives the multiplicative constant as claimed in 
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Lemma 23. Let a and m be nonnegative integers, a even and m odd. Then 

( (c + m + i - j + (b - i + j + l) a +m-j l<i<a 

U(n _L ™\ Vf( a \ 2 U(H1=1\ UV m+l t ( m +!)/2 (m-l)/2 

,_ r ir /2 H ( Q + m ) H (2) H (— ) H (— ) rr (t i , n ; n f w , u , 

" 1 ' H( a+ " +1 ) 2 mfa+m - 1)/2 J-l ^ 2 J_ 1 I 2 tft H2 

(m+l)/2 (m-l)/2 a/2-1 

x J] (^ + A;) a/2 (-±l + fc) a/2 J] (6 + c + m + 2A;)«- 2fc 

fc=l fc=l k=l 

a/2-1 m (m-l)/2 

x (6 + c + 2m + 2A;) a - 2fc JJ (6 + c + 2A;) a+m - fc ]] (b + c + 2A;) m " fc . 

fe=l fe=(m+l)/2 fc=l 

(8.14) 

Proof. We proceed analogously to the proof of Lemma [21]. The only difference is the 



parity of m, so we have to check the places in the proof of Lemma |21] where we used 
the fact that m is even. 

Steps 1, 3-5 can be reused verbatim, but the corresponding products are slightly 
different. 

In Step 2 we are now interested in the factors corresponding to odd values of e 
(1 < e < a — 3), because the factors with even e are covered by Steps 3 and 4. 
Step 6 can be reused verbatim. 

The computation of the multiplicative constant is done analogously to Step 7 in 
the proof of Lemma |21. Using again Figure fill we see that the (— l)-enumeration of 



lozenge tilings, related to our determinant via ( ft.lp and Lemma [TB|, for b = c = 1 equals 
(_l)«/2p+i+§) + (_i)a/2+i ( m+ ^+f- 1 ). It is then a routine computation to verify that 



this gives the multiplicative constant as claimed in (|8.14). □ 



Lemma 24. Let a and m be odd nonnegative integers. Then 

f (c + m + i -j + (b - i +j + l) a+m -j l<i<a 

l<mi + m {(S=*± + i-j + !),_, (*ft* - i + J + l) a+m _ . a < i < a + , 



n (q + i)/2 H (a + m) Hgg) Hgg) H(^) H( — 

/ jj^a+m.-j 2 2m(a+m-l)/2+l/2 

(m+l)/2 (m-l)/2 



II (| + ^)(a-l)/2 11 (f + %+U/2 



X 

fc=l fc=l 

(m+l)/2 (m-l)/2 (o-l)/2 



x n (i+ fc )(«-i)/2 n (i+ fc )(«+D/2 n (&+ c +m+2Ao a - 2fc 

fc=l k=l k=l 

(a-l)/2 m (m-l)/2 

x JJ (6 + c + 2m + 2A;) a " 2fc JJ (b + c + 2k) a+m - k ]"] (6 + c + 2A;) m ~ fc . 

fc=l fc=(m+l)/2 fc=l 

(8.15) 
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Proof. We proceed analogously to the proof of Lemma The parameters a and m 
are odd, so we have to check the places in the proof of Lemma ^1] where we used the 
fact that a or m is even. 

Steps 1-6 can be reused verbatim, but the corresponding products are slightly dif- 
ferent. 

The computation of the multiplicative constant is done analogously to Step 7 in 
the proof of Lemma 17. Again using Figure [H| we see that the (— l)-enumeration of 
lozenge tilings, related to our determinant via ( ft.l| ) and Lemma |T^, for b = c = equals 
(— 1)(° +1 )/ 2 . It is then a routine computation to verify that this gives the multiplicative 
constant as claimed in ( J8.15 ). □ 



9. Proof of Theorem |TT] 

For the proof of Theorem |ll|, we proceed similarly to [27|. We define determinants 
Z n (x,fi) by 

n-l 



Z n (x,fi) 



det 

0<i,j<n-l 



t,k=0 



i + /A fk\ (j — k + fx — 1 



t 



J 



.k-t 



X 



(9.1) 



The only difference to the definition of Z n (x,fi) in p7| is the minus sign in front of 5ij. 
Then an analogue of Theorem 5 of J^J is true. 



Lemma 25. Lei n be a nonnegative integer. Then Z n (x,fi) = if n is odd. If n is 
even, then Z n {x,jj) factors, 



'n-l 



Z n (x,fj) 



-l) n/2 det 

0<«,i<n/2 



\i=0 



j + i\t-i)U-j 



X 



2j+l-i 



'n-l 



x det 

0<«J<n/2- 



t + // + l/z + /i + l 



,t=o 



/i + 1 



t - i 



J 

t-J 



x 



2j-t 



(9.2) 



Proof. As in the proof of Theorem 5 of [27], define matrices S, M, U, 



S 



i + [i 
t 



M 



0<i,t<n-l 



X 



k-t 



0<t,k<n~l 



u 



j — k + n — 1 
j-k 



0<k,j<n-l 



and J and F(x), 
J ----- | ( l)' ' 



— z 



0<i.k<n-l 



J ~ 



J -l J 



(9.3) 



0<i,j<n-l 



Thus, Z n (x,fi) equals det(— 1 + SMU). Now, as in P7| , multiply Z n (x,fi) on the left by 
det(F(l)*) and on the right by det(JF(x)). Subsequently do the manipulations given 
in ]27] (which amount to applying the Chu-Vandermonde summation formula several 
times). The result is that 



Z n {x,fj,)= det (-I + 3MU) 

0<i,j<n—l 



det (-V(x,n) + W(x,fi)), 

0<«j<n— 1 



where 
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V{x^) 2Wj+s = 5>W* ({ ! - V^' (9-4) 

t=o ^ 1 / \ J J 

W(x, »h iwi+ , = f,( t J ( ( _ j _ j***-, (9.5) 



t=0 



where r and s are restricted to be or 1, as in [27]. 

It is straightforward to check that V 2 i i2 j = W 2 ^ 2 j. Hence, each entry of the matrix 
— V + W in an even-numbered row and even-numbered column is 0. This implies that 
det(— V + W) must be whenever the size of the matrix, n, is odd. In the case that n 
is even it implies the factorization 

Z n (x, fi) = det(— V(x, fi) + W(x, fi)) 

= (_l)«/2 det (-V 2ij2j+1 + W 2i , 2j+1 ) det (-V 2i+lj2j + W 2i+l!2j ). 

0<i,j<n/2-l 0<iJ<n/2-l 

As is easily verified, this equation is exactly equivalent to fl9.2|). □ 

Proof of Theorem [77]. Now choose x = 1, /i = m/2, n = a in Lemma Then all the 
sums appearing in ( |9.2| ) can be evaluated by means of the Chu-Vandermonde summa- 
tion ( |7.11|) . The result is 



Z a (l, m /2) = os det_ i (- iy+ ( m+ ;^")) 



(«+?' + m/2)! 

det (3i + m + l) 



o<i,i<o/2-i \ v y (2i - j + m/2)! (2j - i + 1)! 

(i+ j + m/2)\ 



x det (3j + m/2 + l)- 



o<i,j<a/2-i V (2i -j + m/2 + 1)! (2j - 

Both determinants on the right-hand side of this identity can be evaluated by means of 
Theorem 10 in [ I7| , which reads 

(x + y + i+j — l)\ 



det 



o<i,j<n-i \ K ( x + 2i- j) \ (y + 2j - i)\ 



n-l 



i\(x + y + i-l)\(2x + y + 2i) i (x + 2y + 2%), 
JLA (x + 2i)\ (y + 2i)\ ' 1 ' 

This completes the proof of the theorem. □ 



10. Proof of Theorem 12 



We prove Theorem ^ by counting the lozenge tilings of a hexagon with side lengths 
a,a + m,a,a + m,a,a + m and removed central triangle of side length m in two different 
ways. 

First, we already know that this number equals ( |1.2|) with a = b = c. On the other 
hand, we claim that it equals det(J + -B 3 ), where, as before in the proof of Theorem |9| 
in Section [|, B = B(a,m) is the a x a matrix with entries ( m+ j +3 ), < i,j < a — 1, 
and I = 1(a) is the a x a identity matrix. 
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FIGURE 12. A lozenge tiling of a hexagon with a = b = c = 3 and core of size m = 2. 



To prove this claim, we first note that det(/ + -B 3 ) is the sum of all principal minors of 
B 3 . Next we consider the construction used in Section |6] in order to prove polynomiality 
in m of the number of lozenge tilings of a cored hexagon. I.e., we extend all sides of the 
removed triangle to the left (if viewed from the interior of the triangle), as is indicated 
by the thick segments, labeled as Si, S2, and S3, in Figure [12]. These segments cut the 
cored hexagon into three regions. In particular, they cut some of the lozenges in two. 
(In Figure [L2|, these lozenges are shaded.) Subsequently, in each of the three regions, we 
connect the "cut" lozenges by paths, by "following" along the lozenges of the tiling, as 
is illustrated in Figure |1^ by the dashed lines. (Note the difference between Figures [12 
and [5[ In our special case a = b = c all the paths form cycles.) 

Let us number the possible positions of the "cut" lozenges, from inside to outside, 
by 0, 1, . . . , a — 1. Thus, the positions of the "cut" lozenges on the segment Si are 
and 2, they are and 1 on S2, and they are 1 and 2 on 5*3. The number of paths 
in the lower left region which start at position i on Si and end at position j on S2 is 
( m+ j +J ), which is the (i,j)-entry of B. The rotational symmetry of the cored hexagon 
guarantees that an analogous fact is true for the other regions. Thus, the number of 
paths starting at position ion Si, then running around the removed triangle, and finally 
ending at position j on Si, equals the {i, j)-entry of B 3 . If we have a family of paths 
starting and ending at positions ii,i 2 , . . . ,ik, the Lindstrom-Gessel-Viennot theorem 
(see Lemma |T4|) implies that the number of these paths is the minor consisting of rows 
and columns with indices 21,22, • • • ,ik of the matrix B 3 . Thus, the number of these 
families of paths is the sum of all principal minors of B 3 , which we have already found 
to be equal to det(J + B 3 ). 

Now we use the factorization 



/ + B 3 = (/ + B){ul + B)(pl + B), 
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where u is a primitive third root of unity. Thus we have 

det(I + B 3 ) = det(J + B) ■ | de%{ul + B)\ 2 . (10.1) 

The left-hand side equals (|1.2| ) with a = b = c by the above considerations, and the 
determinant det(J + B) has been computed by Andrews [§, Theorem 8], restated here 
as Theorem |10[ 

Thus, a combination of ( |10.1| ), Theorem |1^ and (|1.2|) with a = b = c will give det(ul+ 
B), the determinant that we want to compute, up to a complex factor of modulus 1. We 
note that the determinant is a polynomial in m. It is a routine computation to verify 



that the determinant is the expression claimed in Theorem [12], up to this multiplicative 
constant. 

In order to compute the multiplicative constant, we compute the leading coefficient 
of the determinant as a polynomial in m/2, and compare the result with the leading 



coefficient of the right-hand side of (H^). Unfortunately, the leading coefficient of the 



determinant cannot be determined straightforwardly by extracting the leading coeffi- 
cient of each of the entries and computing the corresponding determinant, for the result 
would be zero. Therefore we have to perform some manipulations of the matrix first to 
avoid cancellation of leading terms. We use the strategy from [E7|], which we have al- 



ready used in the proof of Lemma £5| Instead of the determinant Z n (x, /i), we consider 
here the slightly different determinant 

= os d ILi + g c \ ") (*) p - ; :r y-) . ™ 

where w is a primitive third root of unity. 

Now we proceed analogously to the proof of Lemma [25], i.e., we multiply Z n (x, /i) on 
the left by det(F(l)*) and on the right by det(JF(x)), where the matrices F(x) and J 
are given in (|9.3|) , and use Chu-Vandermonde summation several times. This yields 

Z n (x, /i) = det (ujV(x, h) + W(x, (i)), 

0<i ,j<n— 1 

where V(x,fi) and W(x, /i) are the matrices defined in equation (|9.4j ). 

Now let x = 1, // = m/2, n = a, and V = V(l, m/2), W = W(l, m/2). Again 
using Chu-Vandermonde summation, we can express the desired determinant in terms 
of V = (Vy) <ij< o -i and W = {Wij) <i j<a-i: 

det(uI + B) =det{uV + W), (10.3) 

where 

V 2i+r , 2j+s - (-1) I g + 2j _ i J (10.4) 



W 2WJ+S =[ i + i +m/2 ), (10.5) 



and 



v s + 2j — % — r y 

where r and s are restricted to be or 1. Next we extract the leading coefficients of all 
the entries of ujV + W, viewed as polynomials in m/2, and compute the corresponding 
determinant. If we should obtain something nonzero, then this must be the leading 
coefficient of the determinant det(u>V + W), and hence of det(ul + B), as a polynomial 
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in m/2. Thus, we have to compute the determinant of the matrix L = (Ljj)o<ij< a -i, 
where 



(-1) 



s + l 



tuj if r 



t _ ) (s+2j-i)r 

{(s+2j-i)\ UJ+ (s+2j-i)\ 11 r U ' 

We add row 1 of L to row 0, row 3 to row 2, etc. In that manner, we obtain the matrix 
V = (L'ij) o<i,j<a—Xt where 

if r = 0, 2i ^ a - 1, 



(s+2j-i)! 

jj - ) (~l) J+1 - . if r - 1 

«-i) 8 "+i) if 2i = a - 1 

(*+2j-i)! 

Clearly, we have det L = det L', and we can take out w from all the rows of V with odd 
row index. We get 

detL = u;L§J detL", 



with the matrix L" = (^-)o<i,i<a-i defined by 



i 



if r = 0, 2i ^ a- 1, 



(s+2j-*)l 

/■// _ J if r - 1 

-^2i+r,2j+s - S (s+21-i)! ' 

((-i) a "+D if 2i = a - 1 

Now we add row of L" to row 1, row 2 to row 3, etc. We obtain the matrix L" 
{L"%<i,j<a-u where 



Jin 

lj 2i+r,2j+s 



(s+2j-i)\ 
2 

(s+2j-i)\ 



((-1)^+1) 

I (s+2j-i)\ 



if r = 0, 2i / a- 1, 
if r = 1, s = 1, 



ifr = l, s = 0, 

if 2i = a — 1. 



We rearrange the rows and columns simultaneously, so that the odd-numbered rows 
and columns come before the even-numbered, respectively. Now we have obtained a 
block matrix with one block formed by the rows and columns with odd indices and the 
other one formed by the rows and columns with even indices. Consequently, we have 



detL = ^L«/2j det det (2i-i)! 

0<M<L^J V(! + 2 J 0<i,i<[^J \(^ + 1 )(2Fi)! 




L(a-2)/2j . L(a-1)/2J 



w La/2j ( ^ + 1) x(aodd) 2 La/2j "Q 1 JJ 



x det ((2j - % + 2)i) det ((2j-i + l) 4 ), 

0<i,j<L(a-2)/2j 0<i,j<L(a~l)/2j 

where we used the notation x(A)—l if ^4 is true and x(-4)=0 otherwise. The two deter- 
minants can be evaluated by special cases of a variant of the Vandermonde determinant 
evaluation which we state in Lemma E§ below. After application of this lemma and 
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some simplification we get 



H(0 2 



(10.6) 



H(2/) 



if a is even, a = 21, and 



2 l2+l u\u + 1) 



H(/)H(/ + 1) 
H(2/ + 1) 



(10.7) 



if a is odd, a = 21 + 1. 



It is routine to check that the leading coefficient of the right-hand side of ( |3.4| ), viewed 
as a polynomial in m/2, is exactly the same. 



This finishes the proof of the theorem. □ 



Lemma 26. Let pi be a monic polynomial of degree i, i — 0, 1, . . . , n. Then 



det ( Pi (X j )) = 11 (Xj-Xi). 




11. Proof of Theorem [13] 



If a is even, a = 21 say, the formula can be derived analogously to Theorem [12|. (The 
derivation of the latter was the subject of the preceding section.) Here, the starting 
point is to do the (—1)- enumeration (as opposed to "ordinary" enumeration) of all the 
lozenge tilings of a hexagon with side lengths a, a + m,a,a + m,a,a + m and removed 
central triangle of side length m in two different ways. 

First, the (— l)-enumeration of these lozenge tilings is given by ( |2.1| ) with a = b = c. 
On the other hand, the arguments given at the beginning of the preceding section, 
suitably modified, show that it also equals det(— / + B 3 ), where B is again the matrix 
from the preceding section. 

Now we use the factorization 



where to is a primitive sixth root of unity. (Note that this equation is the analogue of 
( |10.1| ) in the present context.) By the above considerations, the left-hand side equals 
( p.l|) with a = b = c, and the determinant det(— / + B) is computed in Theorem |IT| . 
This determines det {uil + B) up to a multiplicative constant of modulus 1. It is then 
a routine computation to check that the result agrees with the expression at the right- 
hand side of (|3.5| ), up to a factor of modulus 1. 

In order to determine the multiplicative constant, one proceeds as in the preceding 
section. In fact, the determination of the leading coefficient of the determinant as a 
polynomial in m/2 given there can be used here verbatim, because we treated u like 
an indeterminate in the respective computations. Thus, the leading coefficient is the 
expression in ( |10.6| ), with uo now a primitive sixth root of unity. It is routine to check 
that for a = 21 the right-hand side of ( |3.5| ) has the same leading coefficient as polynomial 



Now let us suppose that a is odd, a = 21 + 1 say. Unfortunately, the above strategy 
of determining the value of det (ul + B) through equation (|1 1 . 1| ) fails miserably here, 
because det(— / + I? 3 ) as well as det(— / + B) are zero in the case of odd a (compare 



det(-I + B 3 ) = det(-J + B) ■ \ det(ul + B)\ 2 , 



(11.1) 



in m/2. 
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Theorems f| and [TTJ) . Therefore we have to find a different line of attack. We approach 
the evaluation of det(ul + B), for odd a, by first transforming the determinant in the 



way we have already done in the proofs of Lemma and of Theorem and by then 
applying once again the "identification of factors" method to evaluate the obtained 
determinant. 



In fact, the manipulations explained in the preceding section that proved ( |10.3| ) 



(which are based on multiplying the relevant matrix to the left and right by suitable 
matrices, as elaborated in the proof of Lemma ^ in Section ^) remain valid in the 
present context, again, because there u is treated like an indeterminate. Therefore we 
have 

det (u;I + B) = det(uV + W), 

where the matrices V = (^j)o<i,j<2« and W = (Wij)o<i,j<2i are again the matrices 
defined by ( |HPD and QlQ^I) . 

Our goal is now to evaluate the determinant of the matrix ujV + W. We denote this 
matrix by X(2l + 1, m/2). The determinant detX(2l + 1, m/2) is a polynomial in m, so 
we can indeed use the "identification of factors" method to compute this determinant. 
Again, there are several steps to be performed. In Steps 1-4 below we prove that the 
right-hand side of ( |3.5| ) does indeed divide the determinant as a polynomial in m. In 
Step 5 we determine the maximal degree of the determinant as a polynomial in m. It 
turns out to be (a 2 — l)/4, which is exactly the degree of the right-hand side of ( |3.5|) 



(for odd a, of course). Therefore the determinant must be equal to the right-hand side 
of (|3.5| ), up to a multiplicative constant. This multiplicative constant is finally found 
to be 1 in Step 6. 

Step 1. n[=o^ _1 (T + 2/ — i + l)i-_2«-i divides the determinant detX(2l + l,m/2). 
Proceeding in the spirit of Step 1(b) in the proof of Lemma pj], we prove this by 
finding, for each linear factor of the product, a linear combination of the columns of 
X(2l + l,m/2) which vanishes if the factor vanishes. To be precise, we claim that for 
m/2 = —31 + k + 3d, d > and 1 < k < I — 2d — 1 the following equation holds: 

k-l 

~ (column (2l-2d-2j-l) of X(2l + 1, -31 + k + 3d)) 

+ (column (21 -2d- 2j - 2) of X(2l + 1, -31 + k + 3c/))) = 0. (11.2) 

If we restrict the left-hand side of this equation to the (2i)th row, and simplify a little 
bit, it becomes 



k-l 

E 

j=Q 



k-l\ ((i-2l + 2d-j + k\ (i-2l + 2d-j + k-r ]] 
j J \\2l-2d-2j -i-l) \ 2l-2d-2j -i-2 ' ' ' ' ' 



It becomes (a; — 1) times the same expression if we restrict to the (2i + l)th row. 

As is seen by inspection, the expression ( |11.3| ) vanishes trivially for k — 1. From now 
on, let k > 1. In order to establish that (|11.3|) vanishes in that case as well, we first 
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rewrite the sum ( |11.3| ) in hypergeometric notation ( |7.6| ): 

(k - 1) (2 + Ad + 2% + k - Al)_ 2 -2d-i+2l 



(21 -2d- %- 1)! 

X 4^3 



L3 



l-A;,|-|,l + d + i-Z,i + d+|-Z ' 
\ , 1 - 2d - i - k + 21, 2 + Ad + 2i + k - AV 



;il.4) 



The hypergeometric summation formula which is relevant here, and as well in the 
subsequent steps, is the following "strange" evaluation of a 7_F 6 -series, due to Gessel 
and Stanton |H| (1.7)] (see also |H| (3.8.14), c = 1, a -> q A , etc., q -> 1]): 



7-^6 



A,l + ±,B,l-B,£,l + A-% + n,-n 
j , 1 + 4 - f , | + 4 + f , 1 + A - F, -A + F - 2n, 1 + A + 2n' 



;i + A) 2n (i + |-f 



2 > 



'1 _|_ A _|_ _B _ F 

>2 2 2 2 



(l + A-F) 2n (l + f - 



+ i + 



where n is a nonnegative integer. If in this formula we let B tend to infinity, we obtain 



5^4 



A, 1 -|- -g, ■j, 2 A -)- 7T, Ti 



1 + A - F, -A + F - 2n, 1 + A + 2n 



■A 



[1 + A) 



2 n. 



[1+A-F) 



;n.5) 



2/i 



In particular, this formula allows us to deduce that the left-hand side of (|11.5|) must 
be zero whenever A is a negative integer. This is seen as follows: Multiply both sides 
of (|1J) by 



-A + F - 2n)_ A (1 + A + 2n). A . 



11.6) 



Then, for a fixed negative integer A, the left-hand side becomes polynomial in n. The 
right-hand side is zero for all n larger than —A/2 because of the presence of the term 
(1 + A)2n- The term ( |11.6|) is nonzero for these values of n, therefore the left-hand side 



of ( |11.5| ) must be zero for these n. Since these are infinitely many n, the left-hand side 
of ( |11.5|) must be in fact zero for all n. (An alternative way to see that the left-hand side 
of ( |11.5| ) vanishes for all negative A is by setting c = 1 in |ll], (5.13)] or [|1(], (3.8.11)], 
then replace a by q A , etc., and finally let q — > 1 and B — > oo.) 

If we use (pT~5|) with A = 1 - k, F = 2d + i - 21 + 2, n = 2d + i + k - 21, together with 
the above remarks, then we get immediately that the 4_F3-series in ( |11.4| ) vanishes for 
k > 1. (It should be noted that, for this choice of parameters, the 5_F 4 -series in ( |11.5| ) 
reduces to the 4 F 3 -series in ( |11.4| ).) Thus, equation ( |11.2| ) is established. 



Step 2. nl=o^( m /2 + 2Z — 1)1-2% divides the determinant. We claim that for m/2 
—31 + k + 3d — 1, d> and 1 < k < I — 2d the following equation holds: 

fc— 1 /1 ,\ 

J2[ ~ J ^(column (21 — 2d - 2j) of X(2l + 1, -31 + k + 3d-l)) 

4— n V 3 J 



j=0 



+ (2uo - 1) ■ (column (21 -2d- 2j - 1) of X(2l + 1, —31 + k + 3d- 1)) 
+ (u - 1) ■ (column (21 -2d- 2j - 2) of X(2l + 1, -3Z + k + 3d - 1))) = 0. (11.7) 
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Restricted to the (2z)th row, the left-hand side of this equation becomes, after a little 
simplification, 

fk - 1\ / fi - 21 + 2d - j + k - 1\ fi - 21 + 2d - j + k - 2 N 
f^ \ 3 )\\ 2l-2d-2j-i ) + V 2l-2d-2j-i-l 

(11.8) 



Clearly, this expression vanishes for k — 1. If k > 1, we write ( 11.8|) in hypergeometric 
notation, to obtain 

(A;-l)(4d + 2z + A;-4/)_ 1 _ 2d _ i+2i 
1 + w 



(2/ - 2d-i)l 



x 4 F 3 



I - |, 2 - 3 2d — i — k + 21, Ad + 2i + k-AV 4: 



;ii.9) 



This time we use ( pT5|) with A = 1 - Jfc, F = 2d + i - 21 + 1, n = 2d + i + k - 21 - 1. 
Together with the remarks accompanying (|11.5 ), this implies immediately that the 
4_F 3 -series in ( [L1.9|) vanishes for k > 1. 

On the other hand, restricted to the (2i + l)th row, the left-hand side of ( |1 1 . 7| ) 
becomes, after a little simplification, 

v—^, fk — 1\ / ( i-2l + 2d- j + k\ fi - 21 + 2d - j + k - 1 
-DE , 2(-2d-2 i - i -l + 



j J \\2l-2d-2j -i-lj \ 21 - 2d - 2j - i - 2 
k - 1\ ffi — 2l + 2d — j + k — l\ fi - 21 + 2d - j + k - 2 



j=0 
k-1 

~ W §V 3 AV 2l-2d-2j-i )^\2l-2d-2j-i-l 

That the first sum vanishes was already shown in Step 1 (compare (|11.3| )), that the 
second sum vanishes was shown just above (compare (|11.8|) ). Thus, equation ( |11.7|) is 
established. 

A short argument shows that the linear combinations of Step 1 are independent of 
the linear combinations of Step 2. Let us denote the columns of X(2l + l,m/2) by 
Co, Ci, . . . , C<ii- In Step 1 we have linear combinations of vectors of the form toCik+x + 
Cik-, whereas in Step 2 we have always linear combinations of vectors of the form 
C2fc+2 + (2uj — l)C2k+i + (uj — 1)6*2*;. If these linear combinations were dependent we 
could use the identity 

(C 2fc+2 + (2co - l)C 2k +i + (w - l)C 2fc ) - (u + l)(ioC 2k +i + C 2k ) = C 2k +2 - 2C 2k} 

and get a linear combination of vectors of the form uC 2k +i + C 2k equal to a nonzero 
real linear combination of the Cj's, which is clearly impossible. 

Step 3. n|=o J_1 (W2 + 3« + 5/2), -2i-i divides the determinant. We claim that for 
m/2 = —k — ^d, d odd, d > 1, and 1 < k < I — d the following equation holds: 

fe-i 



/7 -1 \ 

J2( T ) ((row (2i + 2d) of X(2l + l,-k- frf)) 
i=o V ? ^ 



+ w ■ (row (2i + 2d + 1) of X(2/ + 1, -k - \d)) ) = 0. (11.10) 
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Restricted to the (2j)th column, the left-hand side of this equation becomes, after a 
little simplification, 



fc-i 

E 

i=0 



k-1 

i 



d/2 + j - k + 1 
2 j - i - d 



i - d/2 +j — k 
2j -i-d 



11.11) 



It becomes {u — 1) times the same expression if we restrict to the (2j + l)th column. 

Again, the expression ([LI. 11 ) vanishes trivially for k = 1. In order to establish that 
( PlllQ vanishes for k > 1 as well, we reverse the order of summation, and then write 
the sum in hypergeometric notation. Thus we obtain 



, k {1 - k) {d - 2j) k _ x 

4 ] (2j-d)\(-l + j) d „ 2j+k 4 3 



1 - k 



k 1 



+ l 

'3 3' 2 4^2 2' 

\-±l + l-j,2-d + 2j 



4^2 



k ' 

2 ;4 



;n.i2) 



By ( | 1 . 5|) with 1 = 1 — k, F = 1 — d/2 + j — k, n = j — d/2, together with the remarks 
accompanying ( 11.50 , this implies immediately that the 4_F3-series in ( 11.12 ) vanishes 
for k > 1. Thus, equation ( LL1.10Q is established. 

Step 4- nl=o^ ( m /2 + 3i + 3/2)i_2i divides the determinant. We claim that for m/2 = 
—k — |ci — 1, d even, d > 0, and 1 < k < I — d the following equation holds: 



fc-i 

E 

i=0 



fc-1 



(row (2i + 2d) of X(2Z + 1, -k - \d - §)) 

+ (2 - u) ■ (row (2i + 2d + 1) of X(2Z + 1, -jfe - \d - §)) 

- w • (row (2i + 2d + 2) of X(2/ + 1, -Jfe - U - \))] = 0. (11.13) 



Restricted to the (2j)th column, the left-hand side of this equation becomes, after a 
little simplification, 



fc-i 



:i-2c)^ 



i=0 



k-1 



i - d/2 + j - k + 1/2 
2 j - i - d - 1 



+ 



i - d/2 + j - k - 1/2 
2] - i-d - I 



11.14) 



Again, this expression vanishes trivially for k — 1. If > 1, after reversion of summa- 
tion, the hypergeometric form of ( |1 1 . 14p is 



(l-A0(l + d-2j)- 1H 



(2j-d-l)!(-§-f +j) 1+d _ 2j+fe 



x 4 F 3 



fc 3 



'l 3 fc 3 3 4 d 4 
3 — 3 ' 2 2 ^ ^ ~l~ 2j 



+ 1 

2' 4 4 ~ 2 



2 ;4 



. (11.15) 



Now we use ( |il~5|) with 1 = 1 - jfe, F = 1/2 - d/2 +j - k, n = j - d/2 - 1/2. Together 
with the remarks accompanying ( pl.5 ), this implies immediately that the 4.F3-series in 
( LL115| ) vanishes for k > 1. 
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On the other hand, restricted to the (2j + l)th column, the left-hand side of (|11.13[) 
becomes, after a little simplification, 



1 



fc-i 

E 

fc-i 



k- 1 



+ E 



i=0 



fc-1 



- d/2 + j - k + 1/2 
2j -i-d-1 

i-d/2+j -k + 3/2 
2 j - i - d + 1 



+ 



— d/2 + j — As — 1/2 
2 j - i - d - 1 

d/2+j - A; +1/2 
2 j - i - d + 1 



11.16) 



It was already shown just before that the first sum in ( 11.16Q vanishes (compare (|11.14f )). 
The second sum certainly vanishes for k = 1. To see that it vanishes for k > 1 as well, we 
reverse the order of summation and then convert the sum into hypergeometric notation, 



'-11 



(1-fc) (-l + d-2j)- 



l+fe 



(2j-d + !)!(§-§ + j)_ 1+d _ 2i+fe 



x 4 F 3 



k ± 

i 3 
1 



k 3 



3' 4 
k 1 I d 

3' 2 2 



fc 5 
2' 4 

-j,3-d + 2j-k 



i _i_ 2 

4 ~ 2 



4 ~ 2 



2.4 



11.17) 



Again, by QTX5D , this time with A = 1 - A;, F = 3/2 - d/2 + j - k, n = j - d/2 + 1/2, 
together with the remarks accompanying ( 11. 5|) , it follows immediately that the 4F3- 
series in ( [11.171 ) vanishes for k > 1. Thus, equation ( |11.13| ) is established. 

The linear combinations of Steps 3 and 4 are independent by the argument used at 
the end of Step 2. 

Step 5. Determination of the degree o/det X(2l + 1, m/2) as a polynomial in m. The 
(i,j)-entry of X(2l + l,m/2), viewed as polynomial in m, has the degree j — \i/2\. 
Therefore, the determinant of X(2l + 1, m/2) has degree at most 

21 21 

j=0 



i=0 



1(1 + 1) 



polynomial in m. 

Step 6. Computation of the multiplicative constant. It suffices to compute the leading 
coefficient of the determinant detX(2/ + l,m/2) as a polynomial in m/2. This leading 
coefficient can be computed as the determinant of the leading coefficients of the indi- 
vidual entries. In fact, we already did such a computation at the end of the proof of 



Theorem [L2| in the preceding section, with u a primitive third root of unity instead of 
a primitive sixth root of unity. However, since uj was treated there as an indetermi- 
nate, everything can be used here as well. Thus we obtain the expression ( [10.7] ), with 
u a primitive sixth root of unity. It is then routine to check that for a = 21 + 1 the 
right-hand side of (3.5) has the same leading coefficient as a polynomial in m/2. □ 



12. Comments and open problems 

1) Conjectured further enumeration results. There is overwhelming evidence (through 
computer supported empirical calculations) that there are also "nice" formulas for the 
number of lozenge tilings of a cored hexagon for at least two further locations of the 
core. 

First, let a, b and c have the same parity, and consider a hexagon with side lengths 
a,b + m,c,a + m,b,c + m from which an equilateral triangle of side length m is removed 
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c + m 




c + m 




a. Removal of the triangle which is 
off-center by one "unit" 



b. Removal of the triangle which is 
off-center by 3/2 "units" 



Figure 13. 



which is off-center by "one unit". To be more precise, let again s a be the side of the 
triangle which is parallel to the borders of the hexagon of lengths a and a + m, and 
similarly for Sb and s c . Then the distance of s a to the border of length a + m is the 
same as the distance of the vertex of the triangle opposite to s a to the border of length 
a. The distance of Sb to the border of length b + m exceeds the distance of the vertex of 
the triangle opposite to Sb to the border of length b by two units. Finally, the distance 
of s c to the border of length c + m is two units less than the distance of the vertex of 



the triangle opposite to s c to the border of length c. See Figure [[^.a for an example. 
Then the following seems to be true. 

Conjecture 1. Let a,b,c,m be nonnegative integers, a,b,c having the same parity. 
The number of lozenge tilings of a hexagon with sides a,b + m,c,a + m,b,c + m, with an 
equilateral triangle of side length m removed from the position that was described above 
(see Figure \lj^.a), equals 

1 H(a + m) R(b + m) H(c + m) H(a + b + c + m) 
4 H(a + 6 + m)H(a + c + m)H(6 + c + m) 

H(m + [ ^^± § ±£ l ) H(m + [^^J ) 



x 



m + l)H(s±£ 



m 



l)H(*±s 



x 



H( ft 1 ) H( Til ) H( [f 1 ) H( [f J ) H( L|j ) H( [f j ) 



H(f + [f| ) H(f + [fl ) H(f + [fl ) H(f + [f J ) H(f + [§J ) H(f + LfJ ; 
H(f ) 2 H( 



' m\2 XX ( a+b +m \2 / a+c+m \2 TJ / b+c+m \ 2 



' a+b 
> 2 



H(f + \ gd ^] ) H(f + [s±|±sj ) H 
where Pi (a, b, c, m) is the polynomial given by 



1) H( a+C 1 1 ^ u ''^ 



Pi(a,6,c,m), (12.1) 



Pi (a, 6, c, m) 



l^a + b) (a + c) + 2am if a is even, 

(a + 6) (a + c) + 2(a + b + c + m)m if a is odd. 
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The reader should notice that the only differences between formulas ( |12.1| ) and 
( [f .2| ) are in some hyperfactorials involving (a + 6)/2 and (a + c)/2, in the polyno- 



mial Pi(a, 6, c, m), which does not appear in ( |1.2| ), and in the factor 1/4 in front of 

The second case needs a to have a parity different from b and c. Consider a hexagon 
with side lengths a,b + m,c,a + m,b,c + m from which an equilateral triangle of side 
length m is removed which is off-center by "3/2 units". To be more precise, with s a , 
Sb, s c the sides of the triangle as above, the distance of s a to the border of length a + m 
is the same as the distance of the vertex of the triangle opposite to s a to the border of 
length a, the distance of Sb to the border of length b + m exceeds the distance of the 
vertex of the triangle opposite to Sb to the border of length b by three units, and the 
distance of s c to the border of length c + m is three units less than the distance of the 



vertex of the triangle opposite to s c to the border of length c. See Figure |13[b for an 
example. Then the following seems to be true. 

Conjecture 2. Let a, 6, c, m be nonnegative integers, a of parity different from the 
parity ofb and c. The number of lozenge tilings of a hexagon with sides a,b + m, c, a + 
m, 6, c + m, with an equilateral triangle of side length m removed from the position that 
was described above (see Figure \T^.b), equals 

1 H (a + m) H (6 + m) H (c + m) H (a + b + c + m) 
16 H(a + 6 + m)H(a + c + m)H(6 + c + m) 

H (f ) 2 H (ffl) H ([I]) H ([|D H (Lfj) H (L|j) H (Lfj) 
H (f + [|D H (f + [|D H (f + [fD H (f + Lf J) H (f + [§D H (f + Lfj) 
H ([g] + f ) H ([flgj + f ) H (|s±£] + f ) H I ) 2 

H (f + [^D H (f + [^J ) H ([^j - 1) H ([^1 + 1) H (^) 

x ' 2 — V J\ r \J' rP 2 (a,b,c,m), (12.2) 

H([^J + m -l)H(*±£ + m)H([2±*l + m + l) 1 ' 1 ' 

where the polynomial P2(a,b,c,m) is given by 

' ((a + bf - l)((a + c) 2 - 1) + 4am(a 2 + 2a6 + 6 2 + 2ac + 36c + c 2 

+2am + 36m + 3cm + 2m 2 — 1) if a is even, 

((a + 6) 2 - l)((a + c) 2 - 1) + 4(a + 6 + c + m)m(a 2 + 6c - 1) 

z/a is odd. 



P 2 (a, 6, c, m) 



Again, the reader should notice that the only differences between formulas ( |12.2| ) and 
( p. .3D are in some hyperfactorials involving (a + 6)/2 and (a + c)/2, in the polynomial 
P2{a, 6, c, m), which does not appear in ( |1.3| ), and in the factor 1/16 in front of ( |12.2| ). 

Conjectured results about the (— l)-enumeration of the above two families of lozenge 
tilings could be easily worked out as well, and would have similar appearance, i.e., 
the result would be a quotient of products of many "nice" factors times an irreducible 
polynomial of small degree. However, if one moves the triangle farther away from the 
center, then, for both ordinary and (— l)-enumeration, the irreducible polynomial factor 
seems to grow rather quickly in degree, and is therefore difficult to predict in general. 
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For a proof of Conjectures [1] and 0, one might go through considerations analogous 
to those in Section |5], i.e., convert the lozenge tilings into families of nonintersecting 
lattice paths, and, by means of the Lindstrom-Gessel-Viennot theorem (Lemma H), 
obtain a determinant for the number of lozenge tilings. This determinant, which then 



must be evaluated, is 



/ fb + c + m 



det 

l<i,j<a+m 



b-i+j 

b+c 
2 

■-i+j+e 



Ki < a 



\ 



a + l<i<a + rn 



;i2.3) 



with e — I and e = 3/2, respectively. (The determinants in Lemmas ^ and [16] are the 
respective special cases e = and £ = 1/2 of (|12.3|) .) 

2) A multidimensional analogue of Watson's 3F2- summation, and some variants. 
There is another possible way to approach the evaluation of the determinants in Lem- 
15| and [U| This approach consists of applying Laplace expansion to these de- 



mas 



terminants. More precisely, we write an (a + m) x (a + m) determinant (such as the 
determinant in Lemma [15] or |16|) as a (signed) sum of products of a minor formed of 
elements of the first a rows times the complementary minor formed of elements of the 
last m rows. That is, given an (a + m) x (a + m) matrix M, we write 



det M = ^(-l) sW (det M K ) (det M K , 



12.4) 



where the sum is over all a-element subsets K of {1,2,..., a + m}, where s(K) = 
TlkeK k ~ ("t 1 ) ' M K denotes the submatrix of M determined by the first a rows and 
the columns with indices in K, K' denotes the complement of K in {1, 2, . . . , a + m}, 
and Mri denotes the submatrix of M determined by the last m rows and the columns 
with indices in K' . 

The gain in applying ( |12.4| ) to our determinants in Lemmas [15] and |16] is that the 
entries of the resulting minors which then appear on the right-hand side of (|12.4|) have 
now a uniform definition (in contrast to the original determinants), and can in fact 
easily be evaluated in closed form, by means of the determinant evaluation 



det , ■ , 

X<i,j<n \ \l J j 



A 



IL 



<i<j<n 



(£;-£i)lI? =1 (A + i-l)! 



uz =1 (A-L i+ n)\ nr=i(^-i; 



12.5) 



(This determinant evaluation is easily proved, e.g., by means of a general determinant 
lemma from [16 , Lemma 2.2]; see also 0, Sec. 2.2 and (3.12)]). Thus, on the right-hand 
side of Q12.4 ) we obtain a multiple (hypergeometric) series for our determinants. If an 
evaluation of this multiple sum would appear in the existing literature, then we would 
be immediately done. Unfortunately, this does not seem to be the case. On the other 
hand, we did evaluate the determinants in Sections [7] and ^. Thus, comparison of the 
results with the right-hand side in (|12.4 ) establishes summation theorems for multiple 
hypergeometric series. The summation theorem that results, after some replacement of 
parameters, from the evaluations in Section of the determinant in Lemma |T| is the 
following. 
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Theorem 27. Let a be a positive integer and M be a nonnegative integer. The multiple 
series 



l<fci<fc2<---<fc a l<i<j<a 



equals 



12^2 2 >M/2-a/2 



a/2 

n 



i)! 2 (^+f)L C-f+.-V*, (*-§+<) 



M/2-a/2+l 



'Af/2-o/2 



i (f - ,)! (f - j + 1)! ( f + B - l)^ /2 ^ +1 (| + (l + § - i + f ) 2i _, 

(12.7) 

z/ a and M are even, it equals 



-l) u/ "2 

a/2 



«/2iio 2 -o-oM 



M! a nr =1 (^) t -i 

C M\ a 



a/2 

n 



\2 T 2 2 yAf/2-a/2+l/2 i=l VT * + 2/ ! 



X 



Af/2-a/2+l/2 



M/2-1+1/2 



(f + s-i) 



M/2-i+3/2 



(f + 5)ti (i + f -< + ¥)«-! 

(12.8) 



i/ a zs even and M z's odd, i£ equals 

A/f l a f R — -I- 2^ 

•^-a-aM ^ 2^2^/2-0/2+1/2 



-1 



,M/2 f 



m=i(5)i- 



X 



(¥)!(! + b)„ /2 (! + §-¥) 

<- 1)/2 (i-l)!i!(§), 2 (B-f + 



n 



2 2 /M/2-0/2+1/2 
2 / ^ n . ■ i \ 2 
2 J 



2/M/2-a/2+l/2 



t=l 



(?-7(§+5-a;(t+B)^ (§+¥-<+¥)* 

a odd and M even, and vanishes if both a and M are odd. 



(12.9) 



There are two interesting features of this summation theorem to be observed. First, 
if we set a = 1, the theorem reduces to a terminating case of Watson's 3 F 2 -summation 
(see H, (2.3.3.13); Appendix (111.23)]), 



3^2 



A,C,B 

l+A+C - 1 



IB 



r(i)r(| + s)r(| + | + f)r(| 



2 



C 
2 



r(| + #)r(| + f)r(|-4 + s)r(| 



c 

2 



By 



which is a summation formula which is not so often met. Second, however, the above 
theorem is an unusual multidimensional analogue of Watson's 3 ^-summation, because 
of the term rii<i<j<a(^ — ^j) 2 appearing in the summand. Whereas for series containing 
a term like Yli<i<j< a (^i — fy) (i- e -> ^ ne same term, but without the square) there is now 
an extensive theory of summation and transformation formulas (such a series is called 
a hypergeometric series in U(a) or an A a hypergeometric series), mainly thanks to 
Milne and Gustafson (see for example |14|, ^8], |3(| |34|, and the references contained 



therein), it is only occasionally that series containing the square Y\.i<i<i< a (^i ~~ ^ 
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appear. Most of the time, they arise from series featuring Schur functions (see pO 



Theorem 6] for such an example). However, our Theorem |2?| does not seem to extend 
to a "Schur function theorem." 

The summation theorem that results from the evaluations in Section [§] of the deter- 
minant in Lemma is a variant of the preceding theorem. 

Theorem 28. Let a be a positive integer and M be a nonnegative integer. The multiple 
series 

£ n ft-^ri ;-i ( ,,[Hi^„. («.io) 

l<fci<fc 2 <-<fc„ l<i<j<a i=l % " ^2 2+2 + 2)k t \ zn ^ a *)** 

equals 
(-1) 



C M\ a 



(1 , a _|_ C _ MV 
\2 "i" 2 "r 2 2 /Af/2-o/2 

^ (*-l)! 2 (§), i (§), 

nV"" 2 ~ * 2/Af/2-o/2+l V 2 ~ * 2/Af/2-o/2 (19 

i=l 1 2 "'" -° 2>M/2-i \2^ D 2)M/2-i+l \ 2 + 2 1 + 2 ) 2i 

if a and M are even, it equals 
{ } (i + n + c_My li (M n ( 2 

^2 ~ 2 ~ 2 2 J M/2-a/2-l/2 i=l \T 1 + 2 J ! 



x -TT ^2 I" 2^-1 \° 2 T 1 ;M/2-q/2+l/2 fl2 12) 

A i (I + * - !),-! (1 + B " 1), (1 + B - Ifun-^n (l + f "' + ") 



'M/2-1+1/2 V2 1 2 1 2 /2« 

a/ a zs ei>en and M zs ocM, it equals 

(_1) M / 2 2 a2 - a - aM V 2^2 2) M/2-a/2+l/2 



(f + f) (¥)' (f+5-1) 



M/2-o/2+l/2 



x nkWiz; tt (j±jk. (i±j). 

U + 2+2 2 ;M/2-o/2-l/2 i=l IT U + ° ~ l )M/2-i+l 

(a-l)/2 I 1 ) 2 

v TT ^ 2 ^A//2-a/2+l/2 , , 

11 (a , r 1\ (a 1 R n M\ ^IZ.XOj 

f=l U + 5 - (2 + B ~ 2)i { 2 ~ 1 + tJw+1 
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if a is odd and M is even, and it equals 

(C , M\ (M _ a\\ (a , n _ i\ 
\2 ' 2 ) \ 2 2J' \2 *~ /M/2+1/2 



'_1\Af/2-l/2 2 a 2 -a-aA/ ' V 2 2)M/2-a/2+l 



nu(B)>-i "tT (i-l)H!(f), 2 



fl 4- 2 4- £ _ Mr 11 f M ,-,lM 2 /'a,R i\ 2 

12^2^2 2)M/2-a/2 i=l {— ~ 1 + 2) ' \ 2 + ^ ~ 1 J M/2-i+l/2 

TT V 2 ^ * 2)M/2-a/2 \ 2 ^ 1 2) M/2-o/2+l /-iq-i^X 

i/ 6ot/i a and M are odd. 



In fact, the evaluations in Section § of the determinant in Lemma [16] establish even 



a further variant of Theorem This variant is obtained as follows. Recall (see the 
Introduction) that the determinant in Lemma [16] arose from the case when the parity of 
a was different from that of b and c, so that, in order to have a well-defined enumeration 
problem, we had to adjust the definition of a "central" triangle of the hexagon. What 
we did was to shift the really central triangle by half a unit in parallel to the sides 
of the hexagon of length a and a + m. Now let us suppose that, unlike in that case, 
it is b that has parity different from that of a and c, so that the "central" triangle 
in the sense of the Introduction is the really central triangle shifted by half a unit in 
parallel to the sides of the hexagon of length b and b + m. Clearly, our enumeration 
results in Theorems § and |5| can be still used, we just have to interchange the roles of 
a and b. On the other hand, if we go through the considerations in Section |5| {without 
interchange of the roles of a and b, i.e., starting and end points of the lattice paths 
are chosen on the sides of the hexagon of length a and a + m and on the side of the 
triangle which is parallel), then we obtain a certain determinant, which differs slightly 



from the determinants in Lemmas [L5| and [Tb| Comparison of the enumeration results 
with Laplace expansion ( |12.4| ) of the determinant establishes the following summation 
theorem. 

Theorem 29. Let a be a positive integer and M be a nonnegative integer. The multiple 
series 

£ n ft-M'f u, ( Jm^w 2 ^;-2u (1215) 

l<fci<*a<-<fca 1<»<j<o i=l l ' ^ 2 2^ 2) kl ^ n ^ a z ' k ' 

equals 



_1 W2 9 a 2 -a-aM lvl 1 1 li=l 1-° TT 

\2 +n l )a/2\2 + 2 2 I A4/2-a/2 i=l \ 2 1 ) ' \ 2 



X 



|j (j + |)L (B-J + i- i) M/2 _„ /2 (B-q + i- i) tm _ al2+1 

i=i (f + B - l). _ x (f + B - l) M/2 _ { (| + B - \) M/2 _ i+1 (l + j - i + T^i-i 

(12.16) 
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if a and M are even, it equals 

W2 ^-a-aM M\" ELtl^-l ^ (i - I)! 2 

\ l ) Z (a i C M\ a 11 



V2 ^ 2 2 /Af/2-a/2+l/2 i=l V 2 Z+ 27 ! 



a/2 fC\ /C\ ( f> _ C i _ 3\ f n _ <7 , • _ 1\ 

-p-|- l2 7i-l V2 7i V° 2 ^ 2/ M/2-a/2+l/2 V° 2 ^ 2/ Af/2-a/2+l/2 



n 



»=1 12 T " Vt-1 V2 T X A V2 T 2^ M/2-1+1/2 

(12.17) 



' + 5 " (f + fi " !), (f + fi " l)M/2-m/2 (! + §"<+ f )*-! 



a is even and M is odd, it equals 

f -1 NM/2 ^-a-aAf M! " ^ ~ 2 ~ |j A//2-a/2+l/2 fl^lCgji-l 

\ 2 r U / Af/2-o/2+l/2 V2 " r 2 2 /M/2-o/2+l/2 

>< n 



,=i (f-^)' 2 (f + ^-i)L /2 - l+1 



(a-l)/2 fS_2 + i_lr 

I 2 _ 2JM/2-a/2+l/2 , * 



i/ a is odd and M is even, and it equals 

( _ u M/2+l/2 ^-g-aM M! " ( B ~ 2 + f ~ ^M/2-a/2 Il^lCgW 

v ^ (M _ a\\ (a 1 o _ -A fa , C _ M\ a 

\2 2>- \2^~ JD /M/2+1/2 V2 " r 2 2JM/2-a/2 

{ -£ 12 (■•-!)'" (j + jU (j + j), 

X 11 /A/ . . 1\,2 /„ . „ ^2 



J (M-i + l)!'(f + B-l) 



M/2-1+1/2 



x TT ^7 2 n j T'" /2 n n 5 (12-19) 

if both a and M are odd. 

The reader should observe that, by similar considerations, i.e., by applying Laplace 
expansion (|12.4 ) to (|12.3 ), Conjectures [I] and ^] are equivalent to further variations 
of Theorem To be precise, Conjectures [I] and ^ could be proved by establishing 
summation theorems for the multiple series 

i< kl <h-<K liia ti k Hl-f + i + ^ k A^ + a-l- 2e) ki 

with e = 1 and e = 3/2, respectively. 

3) Are there q-analogues of our results? By "g-analogue" , we mean, as usual, that 
objects x are counted with respect to a weight q w ( x \ where w(x) is some statistic defined 
on the objects. The question of whether there is a g-analogue, say of Theorems [l| and 
H, is motivated by two facts: In the case of m = of Theorems [l| and 0, i.e., if one 
counts lozenge tilings of a hexagon with no triangle removed, or, equivalently, plane 
partitions contained in a given box, there is a well-known g-analogue due to MacMahon 
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25| , Sec. 429; proof in Sec. 494], in which every plane partition P is given the weight 



g' where \P\ denotes the number of "boxes" (points, according to our definition of 
plane partitions in Section |3|) of P. The result is the g-analogue of formula fll.lj) which 
is obtained by replacing all factorials in ( |1 . 1|) by the respective g-factorials. Similarly, in 
the case m = 1, g-analogues of Theorems [1] and ^ can be gleaned from |31], Theorem 3], 
by setting Xi = q\ i = 1, 2, . . . , n+ 1, respectively Xi = q\ i = 1, 2, . . . , n, x n+ \ = 0, and 
using the hook-content formula for the principal specialization of Schur functions (see 
|24| , I, Sec. 3, Ex. 1], ||, Ex. A. 30, (ii)]). The question of whether there are g-analogues 
for arbitrary m remains open. Furthermore, it would be particularly interesting if there 
were a g-analogue of Theorem [Tt] that would specialize for m = to the the statement 
of the Macdonald (ex) conjecture on cyclically symmetric plane partitions (cf. 
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